From patchwork Mon Feb 10 17:43:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968259 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 965AC255E33; Mon, 10 Feb 2025 17:43:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209430; cv=none; b=R8LWZG2u5K487eIznpExHnB5hc2rvN/GIq4IL27G5OCB0zIloRld5NaLyZyOLpVAVUTyCUCSsZaqBa5Xd9LBZ8rbG8YTeO/EgwSyYSQ1mAsNYU7CTByWvK5zX1hofGnWyVe6OkB1ahNEdFtN1Pu+TIkgxIDHvPoRYBLFWTd5CzM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209430; c=relaxed/simple; bh=T6e3c/PMYgXS4OKgRSQdlouBBcdvjHT4HakbLjxQBZQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=u7Hll8lUg1lXqyKAzrzUKJE0lLP3V4SH1EbZ7pBFS0dTk5LHIRl7/6TEJWI69NglrenDzPGmHpm3dCzsE8Wg1VYNG5JpGEKBBO707Ub6zWqoJBcV8O7vIYgk+gRJZbnfTsJAIbwYDjMXvn1Q0lx4R4+t7UKpguDE0+H6FWGQQME= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=B3ItClt9; arc=none smtp.client-ip=209.85.216.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="B3ItClt9" Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2f9bac7699aso6826494a91.1; Mon, 10 Feb 2025 09:43:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209428; x=1739814228; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=97lx6pYZgJTlQmKaYcz/MNGoRDCTIp3Q81k8vsbjFMQ=; b=B3ItClt9F/vHdCEUta6j2ZhLlYNB7uFKqtJj9POw1tuH+Isx9sX8ymwoAn4Eckbe9U SE9frsE6q6GU6cqtbHCI2zMfzUN4n3yoZ6/b6yWVqEidqccl39UDUyXYOJpZdVj0t5+D 35KFpcofJXYM7scrmo5l+RGWt6KLag/xNp7IoOoCA4eJmGaJTkar9DxzF8duM7b2f7j4 YtEuKLdeHXefuB2AVZhWJR8bowVC9nVtslhFpYf/RWX94tlFnv3Pcl/OAhTk3Avb7AvP z2kyNG5skE/lUHGmdKxdrctbhE/vpKQmp12ygWsPSrP4+kAffCgWTv92e/hjkcBY2y4M uTaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209428; x=1739814228; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=97lx6pYZgJTlQmKaYcz/MNGoRDCTIp3Q81k8vsbjFMQ=; b=qldTSme5H1jmRv5cY+4iTLflB4par7ebmI8Qy21duN/GllEQdPAkRprR6iv/4xgXl0 UNM0x9C/Ihvsej33X3UXR1GfuwLLl8l/cS00ZHzGzga5xetZiIVedTPnKQumCfHAg+qV +J/F6B+62U9NNilPBWt8Fa3DxBoxHpYkRU/qASPitn3z15vKpT+UTnaKeVnXMPwOAZa9 pIF+zjB1QuKq1zqwSSiplnQLPTJmRtA+CNnmSMpSXgY6cq/I4TGRHR4Ptb4wK0AjUXGB Jpha9HoUgxkgxFTqWBvmvzeQPOxWx6e5JfpbDtqNqCfDrNXDpneIEsOyP+0bCLUGC7pF k57A== X-Gm-Message-State: AOJu0YwZX5rudiqvF4SHEJTnN2vduYzljcvFnMoU3LAtnMDJ2EIO9ase u7/pek6wC779gwO9FinbMRfHOgMsQhr38rY7F4Ghb0E94LJKMLMgyPLc+8Dt X-Gm-Gg: ASbGncvkAO0Yk4OWvStpPuBS3rnnBk8mAJy3OBv18PYQxseOxf27RTqnhEQRS6L/1Jk 9ZoaMnFnlAE0wyaqLq/uH+THM5BSsUqH6mo1ZGNKChjKDQnNGdARTEZ0GMJcyZw2WvpXpJc5yrv BEnTPQ24aPEI3ln4+7H/w7A4FyCEvIg/22+uvli/3mDwAXLtsEmRO+gjoFB6YQTkH+sO1AH6P4O SWzDJrlYsFfZbZu2v8nD3xZopO9lTICsLne5OPolbGUgemJOXTdc+wkuJyw7X+pU5l2FMIdZ8J3 lRKgCziKOADvOAmbUogQXlvkEYU8k70KTgUFvAoEDyZLPJzQM3dMSEeKrzUPCxdMwA== X-Google-Smtp-Source: AGHT+IFIO3TkyEZrtc6D2jlRpAD3Rc/qSd+vcW8XfYQ5NgtcW76LUcNKoN7UcfkqwMTceJfcPIpG/A== X-Received: by 2002:a17:90b:548b:b0:2f4:47fc:7f18 with SMTP id 98e67ed59e1d1-2fa9ed7b2b1mr626002a91.10.1739209427538; Mon, 10 Feb 2025 09:43:47 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.43.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:43:47 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 01/19] bpf: Make every prog keep a copy of ctx_arg_info Date: Mon, 10 Feb 2025 09:43:15 -0800 Message-ID: <20250210174336.2024258-2-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Currently, ctx_arg_info is read-only in the view of the verifier since it is shared among programs of the same attach type. Make each program have their own copy of ctx_arg_info so that we can use it to store program specific information. In the next patch where we support acquiring a referenced kptr through a struct_ops argument tagged with "__ref", ctx_arg_info->ref_obj_id will be used to store the unique reference object id of the argument. This avoids creating a requirement in the verifier that "__ref" tagged arguments must be the first set of references acquired [0]. [0] https://lore.kernel.org/bpf/20241220195619.2022866-2-amery.hung@gmail.com/ Signed-off-by: Amery Hung Acked-by: Eduard Zingerman --- include/linux/bpf.h | 7 +++++-- kernel/bpf/bpf_iter.c | 13 ++++++------- kernel/bpf/verifier.c | 25 +++++++++++++++---------- 3 files changed, 26 insertions(+), 19 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f3f50e29d639..f4df39e8c735 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1507,7 +1507,7 @@ struct bpf_prog_aux { u32 max_rdonly_access; u32 max_rdwr_access; struct btf *attach_btf; - const struct bpf_ctx_arg_aux *ctx_arg_info; + struct bpf_ctx_arg_aux *ctx_arg_info; void __percpu *priv_stack_ptr; struct mutex dst_mutex; /* protects dst_* pointers below, *after* prog becomes visible */ struct bpf_prog *dst_prog; @@ -1945,6 +1945,9 @@ static inline void bpf_struct_ops_desc_release(struct bpf_struct_ops_desc *st_op #endif +int bpf_prog_ctx_arg_info_init(struct bpf_prog *prog, + const struct bpf_ctx_arg_aux *info, u32 cnt); + #if defined(CONFIG_CGROUP_BPF) && defined(CONFIG_BPF_LSM) int bpf_trampoline_link_cgroup_shim(struct bpf_prog *prog, int cgroup_atype); @@ -2546,7 +2549,7 @@ struct bpf_iter__bpf_map_elem { int bpf_iter_reg_target(const struct bpf_iter_reg *reg_info); void bpf_iter_unreg_target(const struct bpf_iter_reg *reg_info); -bool bpf_iter_prog_supported(struct bpf_prog *prog); +int bpf_iter_prog_supported(struct bpf_prog *prog); const struct bpf_func_proto * bpf_iter_get_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog); int bpf_iter_link_attach(const union bpf_attr *attr, bpfptr_t uattr, struct bpf_prog *prog); diff --git a/kernel/bpf/bpf_iter.c b/kernel/bpf/bpf_iter.c index 106735145948..380e9a7cac75 100644 --- a/kernel/bpf/bpf_iter.c +++ b/kernel/bpf/bpf_iter.c @@ -335,7 +335,7 @@ static void cache_btf_id(struct bpf_iter_target_info *tinfo, tinfo->btf_id = prog->aux->attach_btf_id; } -bool bpf_iter_prog_supported(struct bpf_prog *prog) +int bpf_iter_prog_supported(struct bpf_prog *prog) { const char *attach_fname = prog->aux->attach_func_name; struct bpf_iter_target_info *tinfo = NULL, *iter; @@ -344,7 +344,7 @@ bool bpf_iter_prog_supported(struct bpf_prog *prog) int prefix_len = strlen(prefix); if (strncmp(attach_fname, prefix, prefix_len)) - return false; + return -EINVAL; mutex_lock(&targets_mutex); list_for_each_entry(iter, &targets, list) { @@ -360,12 +360,11 @@ bool bpf_iter_prog_supported(struct bpf_prog *prog) } mutex_unlock(&targets_mutex); - if (tinfo) { - prog->aux->ctx_arg_info_size = tinfo->reg_info->ctx_arg_info_size; - prog->aux->ctx_arg_info = tinfo->reg_info->ctx_arg_info; - } + if (!tinfo) + return -EINVAL; - return tinfo != NULL; + return bpf_prog_ctx_arg_info_init(prog, tinfo->reg_info->ctx_arg_info, + tinfo->reg_info->ctx_arg_info_size); } const struct bpf_func_proto * diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 9971c03adfd5..a41ba019780f 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -22377,6 +22377,18 @@ static void print_verification_stats(struct bpf_verifier_env *env) env->peak_states, env->longest_mark_read_walk); } +int bpf_prog_ctx_arg_info_init(struct bpf_prog *prog, + const struct bpf_ctx_arg_aux *info, u32 cnt) +{ + prog->aux->ctx_arg_info = kcalloc(cnt, sizeof(*info), GFP_KERNEL); + if (!prog->aux->ctx_arg_info) + return -ENOMEM; + + memcpy(prog->aux->ctx_arg_info, info, sizeof(*info) * cnt); + prog->aux->ctx_arg_info_size = cnt; + return 0; +} + static int check_struct_ops_btf_id(struct bpf_verifier_env *env) { const struct btf_type *t, *func_proto; @@ -22457,17 +22469,12 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env) return -EACCES; } - /* btf_ctx_access() used this to provide argument type info */ - prog->aux->ctx_arg_info = - st_ops_desc->arg_info[member_idx].info; - prog->aux->ctx_arg_info_size = - st_ops_desc->arg_info[member_idx].cnt; - prog->aux->attach_func_proto = func_proto; prog->aux->attach_func_name = mname; env->ops = st_ops->verifier_ops; - return 0; + return bpf_prog_ctx_arg_info_init(prog, st_ops_desc->arg_info[member_idx].info, + st_ops_desc->arg_info[member_idx].cnt); } #define SECURITY_PREFIX "security_" @@ -22917,9 +22924,7 @@ static int check_attach_btf_id(struct bpf_verifier_env *env) prog->aux->attach_btf_trace = true; return 0; } else if (prog->expected_attach_type == BPF_TRACE_ITER) { - if (!bpf_iter_prog_supported(prog)) - return -EINVAL; - return 0; + return bpf_iter_prog_supported(prog); } if (prog->type == BPF_PROG_TYPE_LSM) { From patchwork Mon Feb 10 17:43:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968260 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A78C7255E44; Mon, 10 Feb 2025 17:43:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209431; cv=none; b=Z0eKPGYgvwVNqk3d4mUrbV5oQRnYX/HctxCPqwciVq6jfusz/7rtgPOWnUQhbnJ3Nm9QuuEc8AmcqIbq3oIdaYdXiJGjiMovhk7jPqxa9Emflm8D1KCn1LwWBambl2XVmKv5lg0PeEnQzqwqJNjjOhfEQJY3BgxIGhkIGqBMU5w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209431; c=relaxed/simple; bh=yeZJv/5oACo15YEXXmcTwVJDy89oXjHZmA6y15XJmEc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pRm1f9+l1Pv1LBXT8+ZQtwSZrwkaLkO3P5JksaJmquwo3m9jWlQB7SUBf1/h4VC48eqwutjhurRX0f4bCKc0z2fNJHVXdNfbcuvwCbLoYmdkl3qHM72oBCS+SE3wcLM8PmYSdFO796iV3uVpjsI3RzjohV8Pl5X1Cmi1gR5K8Xc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YXgebGcq; arc=none smtp.client-ip=209.85.216.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YXgebGcq" Received: by mail-pj1-f53.google.com with SMTP id 98e67ed59e1d1-2fa1e25e337so5620140a91.1; Mon, 10 Feb 2025 09:43:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209429; x=1739814229; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eol6oaMq93kxYk9VK5G7LDWWkiciy0WsGg/oN96JGJ0=; b=YXgebGcqPivO4uqF+gA0Y+DghHOJKxIOHl3TmVj/sKvVqnlWF5Og+GuGHthTe7rbpN J/pcP1hZqY3gjwoRMPHAxpi/j389dv2n1tFAPaDYQs/qDPUC8vCP9GzxQsI3egDrkdrY tRmQOaTtlL70VbUL3aRpmvJuGR15YwaAGW0K7Tkyrm3asSU/Lw6sI+GF0olLpyKZshOs XqS9hRZJnLtApAjkZ0SWU0U5/SxAPe+qiordjwKOYOlsrZxH4gwGvQ8JNG9pIsX9CueT 7XZBzcyAxJRjN2ujoU17nN2ozQMRFf93BBVthKVlGvvNBLFGXJCJ9B71tvB7He4Dbia8 YvOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209429; x=1739814229; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eol6oaMq93kxYk9VK5G7LDWWkiciy0WsGg/oN96JGJ0=; b=vBK5HiuGDvwWDYKbt9CtpTQBg9CKuFYR26AnMqZ4c46P0xh1NEfaq14wfnHr8tg+3p WVD8UVPwnBuSrvjkNTTpkqrE38PZo89pYqfpkqwYmeClZNQgb2z/QEo1QnBMvtQag+tZ QZAJyB4LUDcU6d7qFOO2RzvOJC5eRl6RfE6oTTpg0cRiiUn0EnSmYf7ruJb99gq9oHLj A3jOltO1zzGhaQGZyK+ceuOGjYb+dUe+O5WUy62ef70Pnvqs7j3n627kUf+InrJShZcI ILGHcfNuj3Tmx+PDOrfaEd8OT98zAi+b5OQW2rVkhRjXdPo8EsQQdEAIIj3nrrCIGUgP jzFQ== X-Gm-Message-State: AOJu0Yy2/rhE/6n8DKxYJhrQWcLkoxgYEcPDYE6I7gPKq209mAuN5Lhi ONanHavb2l01a4k5oQBq1Q8p/Tw+d5MpiLwCkrpjUh/wyWAnZPk4AlBVPIPO X-Gm-Gg: ASbGncuQ0UBu0e1ayD6xxiQjMQUgYPr54DjWTXEqZ9VVkXTMgYbVBPmTHyUIK5eJc7P dR7eKLSZf/j8Q7X4k0X9DluRdX8JoUwvTfkW5LHOW6CCY8pEcR1haOHa4nG1UKVirEDkfszgP6e dQVpArgt8KnMqtuBdwHkY/zj1yK4TyxIEbvQuXvyeFp0YYp/a/4BpzdgpqZv2wPMzOqgRMZU1xS V4YNDbDK/aA1Ulo8QEdhHBotj4at8jr226n0NKgMkvfQeXK6t2yrIE1xI2M8sKPXJkx4y3sBySk Q8haohmSbdEKfTF5GJf+5IzvOcuginoQJSyAL7+X//Da4q/87TUtORoYVrAEaJbonQ== X-Google-Smtp-Source: AGHT+IHSgGlsSo0aoMHwfyGSVGqp0CIqGkVlF/KgpHykwHEdvevhjCkdWGp7AhejMggeDJEXLxYWPA== X-Received: by 2002:a17:90b:3c0e:b0:2f2:ab09:c256 with SMTP id 98e67ed59e1d1-2fa243ee52dmr25009848a91.33.1739209428645; Mon, 10 Feb 2025 09:43:48 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.43.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:43:48 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 02/19] bpf: Support getting referenced kptr from struct_ops argument Date: Mon, 10 Feb 2025 09:43:16 -0800 Message-ID: <20250210174336.2024258-3-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Allows struct_ops programs to acqurie referenced kptrs from arguments by directly reading the argument. The verifier will acquire a reference for struct_ops a argument tagged with "__ref" in the stub function in the beginning of the main program. The user will be able to access the referenced kptr directly by reading the context as long as it has not been released by the program. This new mechanism to acquire referenced kptr (compared to the existing "kfunc with KF_ACQUIRE") is introduced for ergonomic and semantic reasons. In the first use case, Qdisc_ops, an skb is passed to .enqueue in the first argument. This mechanism provides a natural way for users to get a referenced kptr in the .enqueue struct_ops programs and makes sure that a qdisc will always enqueue or drop the skb. Signed-off-by: Amery Hung Acked-by: Eduard Zingerman --- include/linux/bpf.h | 3 +++ kernel/bpf/bpf_struct_ops.c | 26 ++++++++++++++++++++------ kernel/bpf/btf.c | 1 + kernel/bpf/verifier.c | 35 ++++++++++++++++++++++++++++++++--- 4 files changed, 56 insertions(+), 9 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f4df39e8c735..15164787ce7f 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -968,6 +968,7 @@ struct bpf_insn_access_aux { struct { struct btf *btf; u32 btf_id; + u32 ref_obj_id; }; }; struct bpf_verifier_log *log; /* for verbose logs */ @@ -1481,6 +1482,8 @@ struct bpf_ctx_arg_aux { enum bpf_reg_type reg_type; struct btf *btf; u32 btf_id; + u32 ref_obj_id; + bool refcounted; }; struct btf_mod_pair { diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 9b7f3b9c5262..68df8d8b6db3 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -146,6 +146,7 @@ void bpf_struct_ops_image_free(void *image) } #define MAYBE_NULL_SUFFIX "__nullable" +#define REFCOUNTED_SUFFIX "__ref" /* Prepare argument info for every nullable argument of a member of a * struct_ops type. @@ -174,11 +175,13 @@ static int prepare_arg_info(struct btf *btf, struct bpf_struct_ops_arg_info *arg_info) { const struct btf_type *stub_func_proto, *pointed_type; + bool is_nullable = false, is_refcounted = false; const struct btf_param *stub_args, *args; struct bpf_ctx_arg_aux *info, *info_buf; u32 nargs, arg_no, info_cnt = 0; char ksym[KSYM_SYMBOL_LEN]; const char *stub_fname; + const char *suffix; s32 stub_func_id; u32 arg_btf_id; int offset; @@ -223,12 +226,19 @@ static int prepare_arg_info(struct btf *btf, info = info_buf; for (arg_no = 0; arg_no < nargs; arg_no++) { /* Skip arguments that is not suffixed with - * "__nullable". + * "__nullable or __ref". */ - if (!btf_param_match_suffix(btf, &stub_args[arg_no], - MAYBE_NULL_SUFFIX)) + is_nullable = btf_param_match_suffix(btf, &stub_args[arg_no], + MAYBE_NULL_SUFFIX); + is_refcounted = btf_param_match_suffix(btf, &stub_args[arg_no], + REFCOUNTED_SUFFIX); + if (!is_nullable && !is_refcounted) continue; + if (is_nullable) + suffix = MAYBE_NULL_SUFFIX; + else if (is_refcounted) + suffix = REFCOUNTED_SUFFIX; /* Should be a pointer to struct */ pointed_type = btf_type_resolve_ptr(btf, args[arg_no].type, @@ -236,7 +246,7 @@ static int prepare_arg_info(struct btf *btf, if (!pointed_type || !btf_type_is_struct(pointed_type)) { pr_warn("stub function %s has %s tagging to an unsupported type\n", - stub_fname, MAYBE_NULL_SUFFIX); + stub_fname, suffix); goto err_out; } @@ -254,11 +264,15 @@ static int prepare_arg_info(struct btf *btf, } /* Fill the information of the new argument */ - info->reg_type = - PTR_TRUSTED | PTR_TO_BTF_ID | PTR_MAYBE_NULL; info->btf_id = arg_btf_id; info->btf = btf; info->offset = offset; + if (is_nullable) { + info->reg_type = PTR_TRUSTED | PTR_TO_BTF_ID | PTR_MAYBE_NULL; + } else if (is_refcounted) { + info->reg_type = PTR_TRUSTED | PTR_TO_BTF_ID; + info->refcounted = true; + } info++; info_cnt++; diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 9de6acddd479..fd3470fbd144 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6677,6 +6677,7 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type, info->reg_type = ctx_arg_info->reg_type; info->btf = ctx_arg_info->btf ? : btf_vmlinux; info->btf_id = ctx_arg_info->btf_id; + info->ref_obj_id = ctx_arg_info->refcounted ? ctx_arg_info->ref_obj_id : 0; return true; } } diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index a41ba019780f..a0f51903e977 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1543,6 +1543,17 @@ static void release_reference_state(struct bpf_verifier_state *state, int idx) return; } +static bool find_reference_state(struct bpf_verifier_state *state, int ptr_id) +{ + int i; + + for (i = 0; i < state->acquired_refs; i++) + if (state->refs[i].id == ptr_id) + return true; + + return false; +} + static int release_lock_state(struct bpf_verifier_state *state, int type, int id, void *ptr) { int i; @@ -5981,7 +5992,8 @@ static int check_packet_access(struct bpf_verifier_env *env, u32 regno, int off, /* check access to 'struct bpf_context' fields. Supports fixed offsets only */ static int check_ctx_access(struct bpf_verifier_env *env, int insn_idx, int off, int size, enum bpf_access_type t, enum bpf_reg_type *reg_type, - struct btf **btf, u32 *btf_id, bool *is_retval, bool is_ldsx) + struct btf **btf, u32 *btf_id, bool *is_retval, bool is_ldsx, + u32 *ref_obj_id) { struct bpf_insn_access_aux info = { .reg_type = *reg_type, @@ -6003,8 +6015,16 @@ static int check_ctx_access(struct bpf_verifier_env *env, int insn_idx, int off, *is_retval = info.is_retval; if (base_type(*reg_type) == PTR_TO_BTF_ID) { + if (info.ref_obj_id && + !find_reference_state(env->cur_state, info.ref_obj_id)) { + verbose(env, "invalid bpf_context access off=%d. Reference may already be released\n", + off); + return -EACCES; + } + *btf = info.btf; *btf_id = info.btf_id; + *ref_obj_id = info.ref_obj_id; } else { env->insn_aux_data[insn_idx].ctx_field_size = info.ctx_field_size; } @@ -7367,7 +7387,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn struct bpf_retval_range range; enum bpf_reg_type reg_type = SCALAR_VALUE; struct btf *btf = NULL; - u32 btf_id = 0; + u32 btf_id = 0, ref_obj_id = 0; if (t == BPF_WRITE && value_regno >= 0 && is_pointer_value(env, value_regno)) { @@ -7380,7 +7400,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn return err; err = check_ctx_access(env, insn_idx, off, size, t, ®_type, &btf, - &btf_id, &is_retval, is_ldsx); + &btf_id, &is_retval, is_ldsx, &ref_obj_id); if (err) verbose_linfo(env, insn_idx, "; "); if (!err && t == BPF_READ && value_regno >= 0) { @@ -7411,6 +7431,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn if (base_type(reg_type) == PTR_TO_BTF_ID) { regs[value_regno].btf = btf; regs[value_regno].btf_id = btf_id; + regs[value_regno].ref_obj_id = ref_obj_id; } } regs[value_regno].type = reg_type; @@ -22148,6 +22169,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog) { bool pop_log = !(env->log.level & BPF_LOG_LEVEL2); struct bpf_subprog_info *sub = subprog_info(env, subprog); + struct bpf_prog_aux *aux = env->prog->aux; struct bpf_verifier_state *state; struct bpf_reg_state *regs; int ret, i; @@ -22255,6 +22277,13 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog) mark_reg_known_zero(env, regs, BPF_REG_1); } + /* Acquire references for struct_ops program arguments tagged with "__ref" */ + if (!subprog && env->prog->type == BPF_PROG_TYPE_STRUCT_OPS) { + for (i = 0; i < aux->ctx_arg_info_size; i++) + aux->ctx_arg_info[i].ref_obj_id = aux->ctx_arg_info[i].refcounted ? + acquire_reference(env, 0) : 0; + } + ret = do_check(env); out: /* check for NULL is necessary, since cur_state can be freed inside From patchwork Mon Feb 10 17:43:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968261 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A1EB92566C0; Mon, 10 Feb 2025 17:43:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209432; cv=none; b=hMdAIp8u3TeSR+AR5xEXexPXtyWfHj/LMSxX6hXSWZpIcyPEvOMnCiMYiUlxA9tYMhgSYYs8KAzz7vkbNII1i+00UZF+JKxPFmuABuPfpu9AHUyURgP9xMIytDBUW15wG/owXDDCehdvmO/Cs2cjmkaGGOP9PwanCFQ0AxYgBb8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209432; c=relaxed/simple; bh=cu4b3Vbm9gFq/EkCNcqo0m8n8rFskmSadxY4u/enMps=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IKuJ30pag7tu07ajIHkC9tUgQAE3RauYhxPwJN6Jv+tFtgI8OH8CMzo/iN9FOqDm4I8gmXvQmEhmahJCn1UeoJ8pIwn/+WMlk8qaKiihEYLaoOahmO5DWIfflpHha3h9f482kNosk5DIywM/HTVlR+bCA5oYnIvEd7xeasKZRcI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=j7+nG4su; arc=none smtp.client-ip=209.85.216.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="j7+nG4su" Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-2f441791e40so6426644a91.3; Mon, 10 Feb 2025 09:43:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209430; x=1739814230; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cB3GDDoQAzmyBzIZYs9Df0cqC8v4dqCoGTbbm/k4J8w=; b=j7+nG4su0VK19Wr1+jFpdry8LLiFxh7mGddIekNzE19V605isUMlCF1VuzBdYREpMT hyKbRHR5HhSwstf6Pj+7CcpK3unZTazipEnbYGmI5v6R93TKw9EaVtIzNxtZSH/3qwCm KS1cMTtNfY2nk8e0sCQj4Oh5F+HyaJTpUUUhtD0EVNSaUCLjYRz800yTWGFOfBXI7Lzm 1S5A5sW3e6p2IgKsL4VE08NZ/KTcJbl9OlowlhyBkM/xJp46FcctJaOl0RRxZP7NTGLv n1DwR/ZVYLtJfva5ImRAYlB24gcY3jb0YovU6SB1+DtsbECaV1BdCHM4w/zYNHOsIez0 UIkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209430; x=1739814230; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cB3GDDoQAzmyBzIZYs9Df0cqC8v4dqCoGTbbm/k4J8w=; b=hX3mO/iuqdV5Gl+EpNMqlbQ/fogSQHOnhT3515JCWKxN1qS+BsAMra1Yj/Tgbh9Pg+ GNK5+ecXP3H2b0XQL29azHFPCnbNJF6+bKhYzyZff5v3YyKAjif68hTlWUB7rTOE3ycP b2fOqwQrMFl74/EcncKgX9UzHiTdRPJKPja3t+ASq+IPzn6+ngnzbUmf8FQXp+Frk1Pz nj/yAnZaBQxyN0VFTjKhC2bnzje38NgfLN+R5a951AazJoC9rF8vwrtufcjoQVpQICin hztaFG0pPN4xhFYvnCTm074Q2R+fkb13v23OWO7TWh4Xy3n4/vVKSCzLCY0vzdgrNL00 w9EQ== X-Gm-Message-State: AOJu0YyxGNF/4B7OxhWWVRkICty/Xh093dwvN/JHjrJZdQLHoKWjL62j pjcB3C27Iw+eyS3fI2RxShHBM9lw13kgGwsdP5gW1P9Hd44ox3Hj/KSjwPDG X-Gm-Gg: ASbGncshakjE/7iE/MuBuzbT5XUi/MzZoBi9fLGpG3Z9yY4dsxnBoXgEsBcErrhVBt5 Vxz5hSUaFL4lqamqziq8od7h54w97oyKcIyZPyl1L88UAms65Q6VBc9PL4lc7W1Qo3+Qcb5o8r/ PNpxiSb4Du5cSoTBf+fKlvK8OiZrNM88uF+fC3U8/1eRpNjs68MNtlBYhrLNRyMTjCwpYKnk1wk vTvDCB16aBMd5qcQdUabLrtj+Hk51tBklu37uWBeNXg4K6PBbkT6jgsiv9P2hH6oaYFyLlW+aYl zz7XmEQhx/y8YROu2vHYWzXMhNn1ZYlhlV0EHLSJCsq6k8uwkkARt5rV/VwJ8pBaJw== X-Google-Smtp-Source: AGHT+IGXVDAdZN1h/noU242b6kA5t3VYz10TPTNjP9NCttGL/LahHRNlFcfAewGlG13Bn25yZanGEw== X-Received: by 2002:a17:90b:2c86:b0:2f5:88bb:118 with SMTP id 98e67ed59e1d1-2fa243e10ccmr18748095a91.22.1739209429736; Mon, 10 Feb 2025 09:43:49 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.43.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:43:49 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 03/19] selftests/bpf: Test referenced kptr arguments of struct_ops programs Date: Mon, 10 Feb 2025 09:43:17 -0800 Message-ID: <20250210174336.2024258-4-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Test referenced kptr acquired through struct_ops argument tagged with "__ref". The success case checks whether 1) a reference to the correct type is acquired, and 2) the referenced kptr argument can be accessed in multiple paths as long as it hasn't been released. In the fail cases, we first confirm that a referenced kptr acquried through a struct_ops argument is not allowed to be leaked. Then, we make sure this new referenced kptr acquiring mechanism does not accidentally allow referenced kptrs to flow into global subprograms through their arguments. Signed-off-by: Amery Hung Acked-by: Eduard Zingerman --- .../prog_tests/test_struct_ops_refcounted.c | 12 ++++++ .../bpf/progs/struct_ops_refcounted.c | 31 +++++++++++++++ ...ruct_ops_refcounted_fail__global_subprog.c | 39 +++++++++++++++++++ .../struct_ops_refcounted_fail__ref_leak.c | 22 +++++++++++ .../selftests/bpf/test_kmods/bpf_testmod.c | 7 ++++ .../selftests/bpf/test_kmods/bpf_testmod.h | 2 + 6 files changed, 113 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_refcounted.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_refcounted.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__global_subprog.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__ref_leak.c diff --git a/tools/testing/selftests/bpf/prog_tests/test_struct_ops_refcounted.c b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_refcounted.c new file mode 100644 index 000000000000..e290a2f6db95 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_refcounted.c @@ -0,0 +1,12 @@ +#include + +#include "struct_ops_refcounted.skel.h" +#include "struct_ops_refcounted_fail__ref_leak.skel.h" +#include "struct_ops_refcounted_fail__global_subprog.skel.h" + +void test_struct_ops_refcounted(void) +{ + RUN_TESTS(struct_ops_refcounted); + RUN_TESTS(struct_ops_refcounted_fail__ref_leak); + RUN_TESTS(struct_ops_refcounted_fail__global_subprog); +} diff --git a/tools/testing/selftests/bpf/progs/struct_ops_refcounted.c b/tools/testing/selftests/bpf/progs/struct_ops_refcounted.c new file mode 100644 index 000000000000..76dcb6089d7f --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_refcounted.c @@ -0,0 +1,31 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +__attribute__((nomerge)) extern void bpf_task_release(struct task_struct *p) __ksym; + +/* This is a test BPF program that uses struct_ops to access a referenced + * kptr argument. This is a test for the verifier to ensure that it + * 1) recongnizes the task as a referenced object (i.e., ref_obj_id > 0), and + * 2) the same reference can be acquired from multiple paths as long as it + * has not been released. + */ +SEC("struct_ops/test_refcounted") +int BPF_PROG(refcounted, int dummy, struct task_struct *task) +{ + if (dummy == 1) + bpf_task_release(task); + else + bpf_task_release(task); + return 0; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_refcounted = { + .test_refcounted = (void *)refcounted, +}; + + diff --git a/tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__global_subprog.c b/tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__global_subprog.c new file mode 100644 index 000000000000..ae074aa62852 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__global_subprog.c @@ -0,0 +1,39 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +extern void bpf_task_release(struct task_struct *p) __ksym; + +__noinline int subprog_release(__u64 *ctx __arg_ctx) +{ + struct task_struct *task = (struct task_struct *)ctx[1]; + int dummy = (int)ctx[0]; + + bpf_task_release(task); + + return dummy + 1; +} + +/* Test that the verifier rejects a program that contains a global + * subprogram with referenced kptr arguments + */ +SEC("struct_ops/test_refcounted") +__failure __log_level(2) +__msg("Validating subprog_release() func#1...") +__msg("invalid bpf_context access off=8. Reference may already be released") +int refcounted_fail__global_subprog(unsigned long long *ctx) +{ + struct task_struct *task = (struct task_struct *)ctx[1]; + + bpf_task_release(task); + + return subprog_release(ctx); +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_ref_acquire = { + .test_refcounted = (void *)refcounted_fail__global_subprog, +}; diff --git a/tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__ref_leak.c b/tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__ref_leak.c new file mode 100644 index 000000000000..e945b1a04294 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__ref_leak.c @@ -0,0 +1,22 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +/* Test that the verifier rejects a program that acquires a referenced + * kptr through context without releasing the reference + */ +SEC("struct_ops/test_refcounted") +__failure __msg("Unreleased reference id=1 alloc_insn=0") +int BPF_PROG(refcounted_fail__ref_leak, int dummy, + struct task_struct *task) +{ + return 0; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_ref_acquire = { + .test_refcounted = (void *)refcounted_fail__ref_leak, +}; diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c index cc9dde507aba..802cbd871035 100644 --- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c @@ -1176,10 +1176,17 @@ static int bpf_testmod_ops__test_maybe_null(int dummy, return 0; } +static int bpf_testmod_ops__test_refcounted(int dummy, + struct task_struct *task__ref) +{ + return 0; +} + static struct bpf_testmod_ops __bpf_testmod_ops = { .test_1 = bpf_testmod_test_1, .test_2 = bpf_testmod_test_2, .test_maybe_null = bpf_testmod_ops__test_maybe_null, + .test_refcounted = bpf_testmod_ops__test_refcounted, }; struct bpf_struct_ops bpf_bpf_testmod_ops = { diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h index 356803d1c10e..c57b2f9dab10 100644 --- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h +++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h @@ -36,6 +36,8 @@ struct bpf_testmod_ops { /* Used to test nullable arguments. */ int (*test_maybe_null)(int dummy, struct task_struct *task); int (*unsupported_ops)(void); + /* Used to test ref_acquired arguments. */ + int (*test_refcounted)(int dummy, struct task_struct *task); /* The following fields are used to test shadow copies. */ char onebyte; From patchwork Mon Feb 10 17:43:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968262 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4CD02566D9; Mon, 10 Feb 2025 17:43:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209433; cv=none; b=CLX+UR/VYlGju8Rp7ul3qSFjYwknLgR6zvsMYO4h36sxrTGUZXvgWfq7ydFT0MDFAGRC1xAwllpZwZ7Nj9o/aUZ+IBqBNaVRcbx1quOaDsuXnvgV9AVO9ad05UzOMh9tYAnx6pKZKA0YB1Fgr1y0iPaheWvlpKYKuDRFEygYPIs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209433; c=relaxed/simple; bh=jwCq0P0Vuds6Ocs62YCkiXqCDb3BXNojP44mv3vZHMM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=n4LAQo/rlAG+Q+nzMhLo9Bj9wy4A+wPGMxN3B3dXBy+XWMZ+jGRERvQaldkEGBYXfojMTl6vb2W6F8Q9E2BpE7pBJcUeQhL4GaYq+aubzEzkkbPqbMLMHa6kZaQJT/g4rlDN8n3ddWnrU7j+3+x6HyD/RZ+R+s8xuHRydYIG0qI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=DI7kF90t; arc=none smtp.client-ip=209.85.216.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DI7kF90t" Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-2fa488351ffso3128745a91.3; Mon, 10 Feb 2025 09:43:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209431; x=1739814231; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=j4LOyoAFByV1hms7BAn0ouvPEgqOVY36PJCCwBmaLhY=; b=DI7kF90tJXzm4LuHhLxH0UpOCL/hMpXIBmmpyPdHKb7zoKZSxMcMR7aWJE78bDlpGj vRXQkF8VsDxCeEqeNcJRpMFB+Qe8iSR5cpwGcfNijcjRnGI11Y84SN4F9tlDQOSlWRWD JhxeTg/Z/sjv9qMK3M3fw90tPT8CH4bw5m9nt29oz0uWr4RO0p5IB2fFMmVAhabaci3d 6OddO+cWvLalKXwE6qYmUpdvhF2lxltlauW0Jmu8MPdZnO5AV+HsU0ix9sm14aqD6X+D Ps56B0vTG6sclM82HC57nZ/D3wIVWcA+U5XhM8c53ckM9rwA8Jod1ysK3kDBMJaeVt2r m5cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209431; x=1739814231; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=j4LOyoAFByV1hms7BAn0ouvPEgqOVY36PJCCwBmaLhY=; b=Dg5j6nTxhuZRLu1CpGDt99+0MyPPwZXVs3AjmpBpujO6ssttxf8xybUDnREkNkJ+em VFeMa6/+p72V01yvInoRacVw+QBX5XnsKUC7hi9NZSJ2K1yk5SosAoH3gAPyAl0b+0Wh aBCRqoKJZA/pQIEcArxl81x6i7/Ay/PoKcelj8Z/gd8O8FEvTH3nZlLgirVW6hFcxyMr RVsCQ+/BJIVLUhnTGhDwOLTWoQ3Ix+rtsAx0ePTS17U3gCgSg601tLnAOxNmkhjqGcCV UJJEeQn9vR6Mbj4JAQLVmkFHoWgzTRoMVUuFgu2rYlhEAPbrvwDQ9+/k2SYKLO9EUoVU 6hnw== X-Gm-Message-State: AOJu0YwMZgkkAJvy46eU3oIyrl7X4wnjjgOWGpaVjOQcn+8UW3q/a7f5 UtOSheDVD8bg1S6F9cN8u5BCJQupF5khUvC6SEwWtRIk8yuMFB/st7Lk7/Pj X-Gm-Gg: ASbGncsTt/pi5H3ubewZMSNQcdXT10LRPhsRvIVZdbGUJotxt2d29+PEeE1EvPlAur3 voUmXhhV9PPWAwj5y64C86rk6x1fiyKqqmKwLZ6Ihx3alc1wK1VtGqtheh2aBXxDgwEvHMJzOw9 6SZM/MA4fk+RfQU5sWwkxoKcDyS5Ewii5Q9BD/zHvGYucCFNwJfYw9QdLHiXtzaPvzUMFvDRiZ8 yMK3tpsp503mL+VMfOfwxIBk0eK4GUO80GAjhU92HJtPzw9SsF5v2mkjV43twI3fvXve6y4XY/p gytdS8t0oD27vD3FbwBKL1Nh/BIYW1U10Eu+cL3cLsiXafd8ZJpvWBk6AJ9vFSV7LQ== X-Google-Smtp-Source: AGHT+IEECGJ+K4Gg6yu4G/m5LgktDngJxq/9vNOgFd9TN9oFeKhAD8AO8p4kEdQCU7tsdvoiZpevmA== X-Received: by 2002:a17:90b:3fc3:b0:2fa:2217:531b with SMTP id 98e67ed59e1d1-2fa2417827dmr20510777a91.21.1739209430787; Mon, 10 Feb 2025 09:43:50 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.43.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:43:50 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 04/19] bpf: Allow struct_ops prog to return referenced kptr Date: Mon, 10 Feb 2025 09:43:18 -0800 Message-ID: <20250210174336.2024258-5-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Allow a struct_ops program to return a referenced kptr if the struct_ops operator's return type is a struct pointer. To make sure the returned pointer continues to be valid in the kernel, several constraints are required: 1) The type of the pointer must matches the return type 2) The pointer originally comes from the kernel (not locally allocated) 3) The pointer is in its unmodified form Implementation wise, a referenced kptr first needs to be allowed to _leak_ in check_reference_leak() if it is in the return register. Then, in check_return_code(), constraints 1-3 are checked. During struct_ops registration, a check is also added to warn about operators with non-struct pointer return. In addition, since the first user, Qdisc_ops::dequeue, allows a NULL pointer to be returned when there is no skb to be dequeued, we will allow a scalar value with value equals to NULL to be returned. In the future when there is a struct_ops user that always expects a valid pointer to be returned from an operator, we may extend tagging to the return value. We can tell the verifier to only allow NULL pointer return if the return value is tagged with MAY_BE_NULL. Signed-off-by: Amery Hung Acked-by: Eduard Zingerman --- kernel/bpf/bpf_struct_ops.c | 12 +++++++++++- kernel/bpf/verifier.c | 36 ++++++++++++++++++++++++++++++++---- 2 files changed, 43 insertions(+), 5 deletions(-) diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 68df8d8b6db3..8df5e8045d07 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -389,7 +389,7 @@ int bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, st_ops_desc->value_type = btf_type_by_id(btf, value_id); for_each_member(i, t, member) { - const struct btf_type *func_proto; + const struct btf_type *func_proto, *ret_type; void **stub_func_addr; u32 moff; @@ -426,6 +426,16 @@ int bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, if (!func_proto || bpf_struct_ops_supported(st_ops, moff)) continue; + if (func_proto->type) { + ret_type = btf_type_resolve_ptr(btf, func_proto->type, NULL); + if (ret_type && !__btf_type_is_struct(ret_type)) { + pr_warn("func ptr %s in struct %s returns non-struct pointer, which is not supported\n", + mname, st_ops->name); + err = -EOPNOTSUPP; + goto errout; + } + } + if (btf_distill_func_proto(log, btf, func_proto, mname, &st_ops->func_models[i])) { diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index a0f51903e977..5bcf095e8d0c 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10758,6 +10758,8 @@ record_func_key(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta, static int check_reference_leak(struct bpf_verifier_env *env, bool exception_exit) { struct bpf_verifier_state *state = env->cur_state; + enum bpf_prog_type type = resolve_prog_type(env->prog); + struct bpf_reg_state *reg = reg_state(env, BPF_REG_0); bool refs_lingering = false; int i; @@ -10767,6 +10769,12 @@ static int check_reference_leak(struct bpf_verifier_env *env, bool exception_exi for (i = 0; i < state->acquired_refs; i++) { if (state->refs[i].type != REF_TYPE_PTR) continue; + /* Allow struct_ops programs to return a referenced kptr back to + * kernel. Type checks are performed later in check_return_code. + */ + if (type == BPF_PROG_TYPE_STRUCT_OPS && !exception_exit && + reg->ref_obj_id == state->refs[i].id) + continue; verbose(env, "Unreleased reference id=%d alloc_insn=%d\n", state->refs[i].id, state->refs[i].insn_idx); refs_lingering = true; @@ -16405,13 +16413,14 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char const char *exit_ctx = "At program exit"; struct tnum enforce_attach_type_range = tnum_unknown; const struct bpf_prog *prog = env->prog; - struct bpf_reg_state *reg; + struct bpf_reg_state *reg = reg_state(env, regno); struct bpf_retval_range range = retval_range(0, 1); enum bpf_prog_type prog_type = resolve_prog_type(env->prog); int err; struct bpf_func_state *frame = env->cur_state->frame[0]; const bool is_subprog = frame->subprogno; bool return_32bit = false; + const struct btf_type *reg_type, *ret_type = NULL; /* LSM and struct_ops func-ptr's return type could be "void" */ if (!is_subprog || frame->in_exception_callback_fn) { @@ -16420,10 +16429,26 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char if (prog->expected_attach_type == BPF_LSM_CGROUP) /* See below, can be 0 or 0-1 depending on hook. */ break; - fallthrough; + if (!prog->aux->attach_func_proto->type) + return 0; + break; case BPF_PROG_TYPE_STRUCT_OPS: if (!prog->aux->attach_func_proto->type) return 0; + + if (frame->in_exception_callback_fn) + break; + + /* Allow a struct_ops program to return a referenced kptr if it + * matches the operator's return type and is in its unmodified + * form. A scalar zero (i.e., a null pointer) is also allowed. + */ + reg_type = reg->btf ? btf_type_by_id(reg->btf, reg->btf_id) : NULL; + ret_type = btf_type_resolve_ptr(prog->aux->attach_btf, + prog->aux->attach_func_proto->type, + NULL); + if (ret_type && ret_type == reg_type && reg->ref_obj_id) + return __check_ptr_off_reg(env, reg, regno, false); break; default: break; @@ -16445,8 +16470,6 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char return -EACCES; } - reg = cur_regs(env) + regno; - if (frame->in_async_callback_fn) { /* enforce return zero from async callbacks like timer */ exit_ctx = "At async callback return"; @@ -16545,6 +16568,11 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char case BPF_PROG_TYPE_NETFILTER: range = retval_range(NF_DROP, NF_ACCEPT); break; + case BPF_PROG_TYPE_STRUCT_OPS: + if (!ret_type) + return 0; + range = retval_range(0, 0); + break; case BPF_PROG_TYPE_EXT: /* freplace program can return anything as its return value * depends on the to-be-replaced kernel func or bpf program. From patchwork Mon Feb 10 17:43:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968263 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E83C32566F7; Mon, 10 Feb 2025 17:43:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209434; cv=none; b=hT1j1Ps8K4nTq+2HoFQS9OQz0HiG0EVKjN4kiI2b7F+pOC1GF5OfhGPeZ16204GTRFM6hp2Bjc6Y6kqJrXbClV3IKEh2cs3R7u+X5nRQi61gpYWlApE+BvFPl9B+aeomHzlhcyxME8LlUdN8T1rY1flwjyzmhydH1H9gC3Jn/pk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209434; c=relaxed/simple; bh=0CPds7L6bVfgfkSwoGmkmAs/hqh/qG7iMp0sGgwerYY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Q3Ktl5XRtPWFd5sqcHpOqGppHei2ZRJiEfTefpZgl+uibquFgztI5CMAz1q0cYC8ZLkfHm9EXhC0uediAoTlW0JWeiJcSqBY4HRTGSAIuAl9UYmwA7FtbLOctwaAXIXjaGzFocg558UjypM9gfFU368JYhkSQSEbHG0cRhC8tS8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ZVzBhNzi; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZVzBhNzi" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-21f48ab13d5so74926735ad.0; Mon, 10 Feb 2025 09:43:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209432; x=1739814232; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aEMmk+QJ7T9vF+Ze9uAr46CKRIACQEkQZAvqN5YNqmw=; b=ZVzBhNziVplqmuWewBolSNgISidkMHdwraaoM2y9qeHBa/ZKb9nsQhwkuAEyarrj7f AxDpj4cPCX8TwSyp29oy9AakqNVNDIiiXFHGaKSASEtmaDCSTw56sJk6g+L5ZW+vSQPn Ckda2pjkR5CD9BZNgdZ+9f0qjsHZBjCru9bt4ewtaCsc8Ezb7oVsO9BLopP6pw8gFnKU 7hnY9LP22Iys7rk5XJfcFunVmEIufN+HPpnQ/yYNbz883jd+xPq2AeFcyGC6MP0REPN1 DdcoKkE7ovexbUg3ZizY35DTnpdmJK1rz5RpUEg7l4qGegng6H2wdKrs1pRq4kDxqb1/ ShFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209432; x=1739814232; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aEMmk+QJ7T9vF+Ze9uAr46CKRIACQEkQZAvqN5YNqmw=; b=Rgum5uyJOiyRtuHATNsuj6yGNX26zfLFQaz3vi+a8F5oxkb4UCy/eWKpK/+iM3T5Fm 52Al6SiQx82a8XdLTp3k4u1f9r96pim7+HSnm9gYf81gGETPN6jsdBDCJFe8Me2awXhq //FgfgBKlv7r7WGJ1zHJls+foWB1JCqoveoUkLaURELLvN3bZkjDPuYoCeamZTkYg7rF L70pNXFvYi626NElekQ+yQqg6299AywwaE7prZAewHlpL45fNT1mK0P3L9nVwfVw9hkv C0z88ULjnKe6PGgg8LiLUBB6MRPYV4+R/h7p/8m9Os9DSIyKFyculMG2P+7fkg2L0rQI YMzw== X-Gm-Message-State: AOJu0YwHu9yoRAQneBP9SCV3TnIlWmeIx4e3j9tzOFGUXgWW3mM0y87R qp8NodWcSMsOcwkY9AfD/MKC2FfcJm8L5d5BFJtUNosLsbgNn5/lukKO0LZK X-Gm-Gg: ASbGncsVLIF7RSqXEYv9XOfsEVvEY5otYkf7ZC4+pSeefKSzLqsI6n/lEZEcHZ19CTN aKbFGbFE//p1OFcSiKu31+RGsNQKeiGiCLcFrHuRlofg9V8qco+I5tnCHaeSlc+bDgHZcAEi6B5 QELQm7jfBAvT3SA9YKxTC45xbeD+9W9TO5lzxeWn5IoIdtnFR37Br18Sh/lXPP+9kEsxQL23dR0 p6bG94S62drZ5HD2idlSwhuKRryyOcD6VPSwRCXAGbabvpyQgDkROs4yOkr8Jtf9xGrGpoUpiCd Uh3w9vvLLPDswqmEqitf6NEftcjrux8rexHIwFdhRWZhkc4+mDSchctY8KI1umF4cA== X-Google-Smtp-Source: AGHT+IERlv+OOVOKrywdfqoHxBOBZTc8CSY6txjHghiULgu81kyLynUEGjTh3ePAlFroyOWv0s1Hiw== X-Received: by 2002:a17:903:947:b0:21b:d2b6:ca7f with SMTP id d9443c01a7336-21f4e75a0aamr244044785ad.32.1739209432026; Mon, 10 Feb 2025 09:43:52 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.43.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:43:51 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 05/19] selftests/bpf: Test returning referenced kptr from struct_ops programs Date: Mon, 10 Feb 2025 09:43:19 -0800 Message-ID: <20250210174336.2024258-6-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Test struct_ops programs returning referenced kptr. When the return type of a struct_ops operator is pointer to struct, the verifier should only allow programs that return a scalar NULL or a non-local kptr with the correct type in its unmodified form. Signed-off-by: Amery Hung Acked-by: Eduard Zingerman --- .../prog_tests/test_struct_ops_kptr_return.c | 16 +++++++++ .../bpf/progs/struct_ops_kptr_return.c | 30 ++++++++++++++++ ...uct_ops_kptr_return_fail__invalid_scalar.c | 26 ++++++++++++++ .../struct_ops_kptr_return_fail__local_kptr.c | 34 +++++++++++++++++++ ...uct_ops_kptr_return_fail__nonzero_offset.c | 25 ++++++++++++++ .../struct_ops_kptr_return_fail__wrong_type.c | 30 ++++++++++++++++ .../selftests/bpf/test_kmods/bpf_testmod.c | 8 +++++ .../selftests/bpf/test_kmods/bpf_testmod.h | 4 +++ 8 files changed, 173 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_kptr_return.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_kptr_return.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__invalid_scalar.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__local_kptr.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__nonzero_offset.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__wrong_type.c diff --git a/tools/testing/selftests/bpf/prog_tests/test_struct_ops_kptr_return.c b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_kptr_return.c new file mode 100644 index 000000000000..467cc72a3588 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_kptr_return.c @@ -0,0 +1,16 @@ +#include + +#include "struct_ops_kptr_return.skel.h" +#include "struct_ops_kptr_return_fail__wrong_type.skel.h" +#include "struct_ops_kptr_return_fail__invalid_scalar.skel.h" +#include "struct_ops_kptr_return_fail__nonzero_offset.skel.h" +#include "struct_ops_kptr_return_fail__local_kptr.skel.h" + +void test_struct_ops_kptr_return(void) +{ + RUN_TESTS(struct_ops_kptr_return); + RUN_TESTS(struct_ops_kptr_return_fail__wrong_type); + RUN_TESTS(struct_ops_kptr_return_fail__invalid_scalar); + RUN_TESTS(struct_ops_kptr_return_fail__nonzero_offset); + RUN_TESTS(struct_ops_kptr_return_fail__local_kptr); +} diff --git a/tools/testing/selftests/bpf/progs/struct_ops_kptr_return.c b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return.c new file mode 100644 index 000000000000..36386b3c23a1 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return.c @@ -0,0 +1,30 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +void bpf_task_release(struct task_struct *p) __ksym; + +/* This test struct_ops BPF programs returning referenced kptr. The verifier should + * allow a referenced kptr or a NULL pointer to be returned. A referenced kptr to task + * here is acquried automatically as the task argument is tagged with "__ref". + */ +SEC("struct_ops/test_return_ref_kptr") +struct task_struct *BPF_PROG(kptr_return, int dummy, + struct task_struct *task, struct cgroup *cgrp) +{ + if (dummy % 2) { + bpf_task_release(task); + return NULL; + } + return task; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_kptr_return = { + .test_return_ref_kptr = (void *)kptr_return, +}; + + diff --git a/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__invalid_scalar.c b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__invalid_scalar.c new file mode 100644 index 000000000000..caeea158ef69 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__invalid_scalar.c @@ -0,0 +1,26 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +struct cgroup *bpf_cgroup_acquire(struct cgroup *p) __ksym; +void bpf_task_release(struct task_struct *p) __ksym; + +/* This test struct_ops BPF programs returning referenced kptr. The verifier should + * reject programs returning a non-zero scalar value. + */ +SEC("struct_ops/test_return_ref_kptr") +__failure __msg("At program exit the register R0 has smin=1 smax=1 should have been in [0, 0]") +struct task_struct *BPF_PROG(kptr_return_fail__invalid_scalar, int dummy, + struct task_struct *task, struct cgroup *cgrp) +{ + bpf_task_release(task); + return (struct task_struct *)1; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_kptr_return = { + .test_return_ref_kptr = (void *)kptr_return_fail__invalid_scalar, +}; diff --git a/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__local_kptr.c b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__local_kptr.c new file mode 100644 index 000000000000..b8b4f05c3d7f --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__local_kptr.c @@ -0,0 +1,34 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_experimental.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +struct cgroup *bpf_cgroup_acquire(struct cgroup *p) __ksym; +void bpf_task_release(struct task_struct *p) __ksym; + +/* This test struct_ops BPF programs returning referenced kptr. The verifier should + * reject programs returning a local kptr. + */ +SEC("struct_ops/test_return_ref_kptr") +__failure __msg("At program exit the register R0 is not a known value (ptr_or_null_)") +struct task_struct *BPF_PROG(kptr_return_fail__local_kptr, int dummy, + struct task_struct *task, struct cgroup *cgrp) +{ + struct task_struct *t; + + bpf_task_release(task); + + t = bpf_obj_new(typeof(*task)); + if (!t) + return NULL; + + return t; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_kptr_return = { + .test_return_ref_kptr = (void *)kptr_return_fail__local_kptr, +}; diff --git a/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__nonzero_offset.c b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__nonzero_offset.c new file mode 100644 index 000000000000..7ddeb28c2329 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__nonzero_offset.c @@ -0,0 +1,25 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +struct cgroup *bpf_cgroup_acquire(struct cgroup *p) __ksym; +void bpf_task_release(struct task_struct *p) __ksym; + +/* This test struct_ops BPF programs returning referenced kptr. The verifier should + * reject programs returning a modified referenced kptr. + */ +SEC("struct_ops/test_return_ref_kptr") +__failure __msg("dereference of modified trusted_ptr_ ptr R0 off={{[0-9]+}} disallowed") +struct task_struct *BPF_PROG(kptr_return_fail__nonzero_offset, int dummy, + struct task_struct *task, struct cgroup *cgrp) +{ + return (struct task_struct *)&task->jobctl; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_kptr_return = { + .test_return_ref_kptr = (void *)kptr_return_fail__nonzero_offset, +}; diff --git a/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__wrong_type.c b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__wrong_type.c new file mode 100644 index 000000000000..6a2dd5367802 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__wrong_type.c @@ -0,0 +1,30 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +struct cgroup *bpf_cgroup_acquire(struct cgroup *p) __ksym; +void bpf_task_release(struct task_struct *p) __ksym; + +/* This test struct_ops BPF programs returning referenced kptr. The verifier should + * reject programs returning a referenced kptr of the wrong type. + */ +SEC("struct_ops/test_return_ref_kptr") +__failure __msg("At program exit the register R0 is not a known value (ptr_or_null_)") +struct task_struct *BPF_PROG(kptr_return_fail__wrong_type, int dummy, + struct task_struct *task, struct cgroup *cgrp) +{ + struct task_struct *ret; + + ret = (struct task_struct *)bpf_cgroup_acquire(cgrp); + bpf_task_release(task); + + return ret; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_kptr_return = { + .test_return_ref_kptr = (void *)kptr_return_fail__wrong_type, +}; diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c index 802cbd871035..89dc502de9d4 100644 --- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c @@ -1182,11 +1182,19 @@ static int bpf_testmod_ops__test_refcounted(int dummy, return 0; } +static struct task_struct * +bpf_testmod_ops__test_return_ref_kptr(int dummy, struct task_struct *task__ref, + struct cgroup *cgrp) +{ + return NULL; +} + static struct bpf_testmod_ops __bpf_testmod_ops = { .test_1 = bpf_testmod_test_1, .test_2 = bpf_testmod_test_2, .test_maybe_null = bpf_testmod_ops__test_maybe_null, .test_refcounted = bpf_testmod_ops__test_refcounted, + .test_return_ref_kptr = bpf_testmod_ops__test_return_ref_kptr, }; struct bpf_struct_ops bpf_bpf_testmod_ops = { diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h index c57b2f9dab10..c9fab51f16e2 100644 --- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h +++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h @@ -6,6 +6,7 @@ #include struct task_struct; +struct cgroup; struct bpf_testmod_test_read_ctx { char *buf; @@ -38,6 +39,9 @@ struct bpf_testmod_ops { int (*unsupported_ops)(void); /* Used to test ref_acquired arguments. */ int (*test_refcounted)(int dummy, struct task_struct *task); + /* Used to test returning referenced kptr. */ + struct task_struct *(*test_return_ref_kptr)(int dummy, struct task_struct *task, + struct cgroup *cgrp); /* The following fields are used to test shadow copies. */ char onebyte; From patchwork Mon Feb 10 17:43:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968264 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2460A25742A; Mon, 10 Feb 2025 17:43:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209435; cv=none; b=e/lG8BwXLFvNbC2dZjjbunCmJHI+aOjHvvbwsnpU+qs7ErGxcsE7ht9jgHKWXVfF4fvFLRY5b7q+R3dIX5JyFc3UybanMOnXVGxX5T7jcOWjDCltgoAVoAZpmOQXXmry+i/CK1VWnV3+eMR7Ihk/ZhUhDAsD9+RcCqqGiV44Ny4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209435; c=relaxed/simple; bh=eB27qpGeHbnL+g+H5i6dwyeK5gEdO4WsfdSVSblw4mA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tzWu+MnqfOGdkD7952RanoSJrk+0VEA4In51X5MiFJgEJ9YhfcvjkD+XBHa88hoepQskpnMS0i/Xs25UHjEceii44Z9qQC68qcLLEOVgpwAxiXL/0kWkrrouDrRilnF5QHz1YkTlYa1QKdDb8f2TOs0iJi7XVCreglydl2ftEus= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=RVE84jvw; arc=none smtp.client-ip=209.85.216.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RVE84jvw" Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-2fa3fe04dd2so3665048a91.0; Mon, 10 Feb 2025 09:43:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209433; x=1739814233; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Bqaa0Rde68BkHu0sINbJqayYMFBgLoQtb5xV4LwmnQE=; b=RVE84jvwHuIPFKQ+++vTf8yB9IGi6L5E3G1AtRR7bvDxAy8DKzSWY7HmuaXV6u6QGH HITDqtV3Jn/qTU7R2PIE/tLtmcIWpQ6pHwfl3G+Tt7Emzr5xLxOtBkRFIOyStWi5pF8g 6neFNEjo9FZpoQx+S4AlFyXMfnYWhiwMuLGgMppCWsmtt0uozeisDeLI5uo5Evdqymc/ I3aJ7V42XMTdjJ1wJPKv62bQVgdjfkf/EShujU/HyiEx/tVSkoNcPxWdQ1yAVrYPKQsj IlTTYkCqPxv3XXtoaQKxX17LxEj4PHibpzBjB/0yKwJi9hIA990hxfWRa152w7pEs73n EGrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209433; x=1739814233; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Bqaa0Rde68BkHu0sINbJqayYMFBgLoQtb5xV4LwmnQE=; b=lG0RfUoX1y1sIM6z5vhe24ZM+wy4+dpjKU7Ae2ZvBAidtUZZgwulsdYGK7hRnT6tAH mVlYq93QoNBPVrh3mkIzA3C1oy0L4GxZI6r7G75a9m4sIvKYbdEZ8Bg7pInjz50158bV SXARr74Uw+nvTatLXl0EXnl9Upj9uYDBQMzkq7pywMYKHVyP0huUXeV/klD1Xi2jlxD+ o/zQXfOYK6nnY96Gkgz5nrwcNMCgzUHLyj866+MSjuNMd8pNgxRXqfaV845XAYEiAtj3 m1AJFH6PuHyaQrTXd3tlonipFyUPzRvCjZCk506V5PDSkz3XR2wfzLT7RLxkeS7h9Z7o v5Pw== X-Gm-Message-State: AOJu0YzkmCvRdSZFuPnQYK0u4yX9BYBrx4PF00F1a7UDjUsSRPXgWpph K5iBCxldyRStt5n7XH1Pd+U0jkaLc4QI1GA9tD5njF5QOL9uoA3F9RwVPGbW X-Gm-Gg: ASbGnctDd8ipBwkfOU7qpQLih/MkcvSTLcy24GAuxjxuOnrRWeUj2JjaUqe3WSdT/iD oD01UEtHXux+ljf3EUvkNRUmIFb8ZYlbHgobzNPNyRAzIoCF5bDYC6d9WOF8LDgCfVIhbF9AGv4 8evpP3G1zs08SLH02J0fK7kJTWux93HO/1jErqjwIyQTY8b3jNYfeWWssKU5BmSMlOr6riN4uDz 49q10EIXFp7awVqDA5Uu6J43sE4mf4srMBMSToQR/031McZ7iz+YGD3Sc4V5yRr01rgg1tPvaUD qRqd9wzjC0Hol0nI3CGdgO/CmiZQoTMK0OfsIYYg2/MZ2DUT14hEhZqPcD4w13X57w== X-Google-Smtp-Source: AGHT+IH7xj9/pLQPU0J7F7gET9vRaqBouDpQVyF9o4nTg17imPuJDQ5E/IOFc7We+274oVRatybPHg== X-Received: by 2002:a17:90b:4c02:b0:2ea:59e3:2d2e with SMTP id 98e67ed59e1d1-2fa2406418emr22983490a91.10.1739209433127; Mon, 10 Feb 2025 09:43:53 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.43.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:43:52 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 06/19] bpf: Prepare to reuse get_ctx_arg_idx Date: Mon, 10 Feb 2025 09:43:20 -0800 Message-ID: <20250210174336.2024258-7-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Rename get_ctx_arg_idx to bpf_ctx_arg_idx, and allow others to call it. No functional change. Signed-off-by: Amery Hung --- include/linux/btf.h | 1 + kernel/bpf/btf.c | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/include/linux/btf.h b/include/linux/btf.h index 2a08a2b55592..ce057c6b3947 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -519,6 +519,7 @@ bool btf_param_match_suffix(const struct btf *btf, const char *suffix); int btf_ctx_arg_offset(const struct btf *btf, const struct btf_type *func_proto, u32 arg_no); +u32 btf_ctx_arg_idx(struct btf *btf, const struct btf_type *func_proto, int off); struct bpf_verifier_log; diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index fd3470fbd144..ca5779f6961b 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6370,8 +6370,8 @@ static bool is_int_ptr(struct btf *btf, const struct btf_type *t) return btf_type_is_int(t); } -static u32 get_ctx_arg_idx(struct btf *btf, const struct btf_type *func_proto, - int off) +u32 btf_ctx_arg_idx(struct btf *btf, const struct btf_type *func_proto, + int off) { const struct btf_param *args; const struct btf_type *t; @@ -6549,7 +6549,7 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type, tname, off); return false; } - arg = get_ctx_arg_idx(btf, t, off); + arg = btf_ctx_arg_idx(btf, t, off); args = (const struct btf_param *)(t + 1); /* if (t == NULL) Fall back to default BPF prog with * MAX_BPF_FUNC_REG_ARGS u64 arguments. From patchwork Mon Feb 10 17:43:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968265 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24194257435; Mon, 10 Feb 2025 17:43:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209436; cv=none; b=F/yRg96vvuky1nxAMHVNnfSlUiyvrLfFyKL6Kz4janlF5zBU+CqDklbyNTroTIlA5nLueh0CBWr4LzVSLZJX4ANy9eaWP3JiHqKsLx0rW67+MACIL2ZgDkfZpGbzXoebqxHq7hoSWlla6jXAwOPUdUDsIPanvSvC1bdi9NkGnpQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209436; c=relaxed/simple; bh=4L0QpGoYr6OkA1ibAzBIDfCnhoaWO/S8JALDmmJDF48=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pFFykS/j2WKjjr4x+u3jTC5dAfBB+iiprywD8f4nj1IjOh4ViRFtghCukCVj8bVsos5pLm0fqvQAxFWVJxtJEoHkZra9En23sxmAL9DS96JofjZ7vx+kAJner98V0pY5PJx5MRdZhu4hhavhxc5uSjZU8rR+d7u4LodFryutXr4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JVCkWQyC; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JVCkWQyC" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-21f6f18b474so31140385ad.1; Mon, 10 Feb 2025 09:43:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209434; x=1739814234; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=asG0Td5Uy+/WENZShXcphGkKxJyJZZ2l2bOrEIfzsQE=; b=JVCkWQyCBqBUP1MkTv28RGYWkLAWVUUR55xCzdR7H2IZkNuwRF7cAA83D8iADS3QzQ quK2xU7vVeQIB8ybUX5udsfXYWuO3pA6DNKl8uhzo8mkKxNej3sZd1aNOdRw6Cbo6v2b AaqKH8ZKsjUKGUCQQCjRbMTUEMgqwW4zdiD3qD3kv4yaeQnHHZdja03zODKu3EC22+JK MkFmpaGIUv+QZIMRZQfdVqvkYyxqxZOXP5Rl/EpxrSxokV4FyjdLuWinDmStmSM/AaDE DXgXSc/ubdogtWOJcaTX1mqg5pyV61CHtnkURIUqpjjq0p8VDy9dI73EjDdeRirm4/Zv EXLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209434; x=1739814234; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=asG0Td5Uy+/WENZShXcphGkKxJyJZZ2l2bOrEIfzsQE=; b=bw9b7DsKp3eK9elhyEXaN+HVq6yxHpu3yyHmbtGBBUJL5v9xmUcMuSwp2uJOO4LTH1 OXfy9mLtX7w/1BuCnfEH3Yvv00s+oo2YQaqdI6NS4j19UY+5mquWlNs0FojwUcUzwJjC nnnbeGEMk0e3yv7CUezo5Uegg/Snram8UeVrshF8W/im85Vf84e7bASPOcfiM4QTemve 2YQXVGdjts80AZgK6WE9ee2Dyyjn2O04VHf29LOmzX06PRUyly3F0KPYawNaDXoe+Myt zgp0gz+wcYWJvGb2IOlqx5N/IzpZLwcPjUaliWgI+3bqxTnkpaHV2fXcuJbf6Xt6yAAT UCyQ== X-Gm-Message-State: AOJu0YzlL1ZXY2rZ8ms92vrNOxAz7cR/asTjUpEDajrwv/4H2V8bTQ5W rzPiTkN5NEjN6Z6O+P/Wdny7VzISlC9u6+KdUJPW3g2tW73tKj7r7MVJ4R49 X-Gm-Gg: ASbGnctO6ZYA1aBxscimfouiPGOhdHLEZW6FAjbnMlk6CpIdiy1cEIOYto1aYZW3Oqm Sl66LYJzFpg/MVGtFMBWC66H83eyfVflHKBDm1kWqJ9EYG6q2sSOn8qCXiI9si1paYLi5PokKHL B0CU3ng3j8DtWLYiI3lEjC9lSl984hTQNyw+b/kFt8VLUDIpG7aWslPqORihMuQHpxSwlygf1eu TNRX8FPIYm8HdoNnojJ51XaLwR0A763rcpB+86usEZHmhc0qaf+ljjMdbv6K9pi3PtZtY3M7Yqm hifg+QhajGxq41vP0oUKpd21Fa+lZ+Uou3DSns1Zdc6Rauf1w10sl6PWw8LwiPXlCA== X-Google-Smtp-Source: AGHT+IEQY8H+wV3mr/kLSqpmpdK10CBNVqKDbw6hZ/ad+anIu50qHjeZGtsocwWzakH1pS1iqVuCPA== X-Received: by 2002:a17:903:228b:b0:216:7926:8d69 with SMTP id d9443c01a7336-21f4e7832d9mr199739875ad.47.1739209434189; Mon, 10 Feb 2025 09:43:54 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.43.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:43:53 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 07/19] bpf: Generalize finding member offset of struct_ops prog Date: Mon, 10 Feb 2025 09:43:21 -0800 Message-ID: <20250210174336.2024258-8-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Generalize prog_ops_moff() so that we can use it to retrieve a struct_ops program's offset for different ops. Signed-off-by: Amery Hung Acked-by: Eduard Zingerman --- include/linux/bpf.h | 1 + kernel/bpf/bpf_struct_ops.c | 13 +++++++++++++ net/ipv4/bpf_tcp_ca.c | 23 ++--------------------- 3 files changed, 16 insertions(+), 21 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 15164787ce7f..6003ba36f6c5 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1892,6 +1892,7 @@ static inline void bpf_module_put(const void *data, struct module *owner) module_put(owner); } int bpf_struct_ops_link_create(union bpf_attr *attr); +u32 bpf_struct_ops_prog_moff(const struct bpf_prog *prog); #ifdef CONFIG_NET /* Define it here to avoid the use of forward declaration */ diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 8df5e8045d07..d3a76f0c5a82 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -1386,3 +1386,16 @@ void bpf_map_struct_ops_info_fill(struct bpf_map_info *info, struct bpf_map *map info->btf_vmlinux_id = btf_obj_id(st_map->btf); } + +u32 bpf_struct_ops_prog_moff(const struct bpf_prog *prog) +{ + const struct btf_member *m; + const struct btf_type *t; + u32 midx; + + t = btf_type_by_id(prog->aux->attach_btf, prog->aux->attach_btf_id); + midx = prog->expected_attach_type; + m = &btf_type_member(t)[midx]; + + return __btf_member_bit_offset(t, m) / 8; +} diff --git a/net/ipv4/bpf_tcp_ca.c b/net/ipv4/bpf_tcp_ca.c index 554804774628..415bd3b18eef 100644 --- a/net/ipv4/bpf_tcp_ca.c +++ b/net/ipv4/bpf_tcp_ca.c @@ -16,7 +16,6 @@ static struct bpf_struct_ops bpf_tcp_congestion_ops; static const struct btf_type *tcp_sock_type; static u32 tcp_sock_id, sock_id; -static const struct btf_type *tcp_congestion_ops_type; static int bpf_tcp_ca_init(struct btf *btf) { @@ -33,11 +32,6 @@ static int bpf_tcp_ca_init(struct btf *btf) tcp_sock_id = type_id; tcp_sock_type = btf_type_by_id(btf, tcp_sock_id); - type_id = btf_find_by_name_kind(btf, "tcp_congestion_ops", BTF_KIND_STRUCT); - if (type_id < 0) - return -EINVAL; - tcp_congestion_ops_type = btf_type_by_id(btf, type_id); - return 0; } @@ -135,19 +129,6 @@ static const struct bpf_func_proto bpf_tcp_send_ack_proto = { .arg2_type = ARG_ANYTHING, }; -static u32 prog_ops_moff(const struct bpf_prog *prog) -{ - const struct btf_member *m; - const struct btf_type *t; - u32 midx; - - midx = prog->expected_attach_type; - t = tcp_congestion_ops_type; - m = &btf_type_member(t)[midx]; - - return __btf_member_bit_offset(t, m) / 8; -} - static const struct bpf_func_proto * bpf_tcp_ca_get_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) @@ -166,7 +147,7 @@ bpf_tcp_ca_get_func_proto(enum bpf_func_id func_id, * setsockopt() to make further changes which * may potentially allocate new resources. */ - if (prog_ops_moff(prog) != + if (bpf_struct_ops_prog_moff(prog) != offsetof(struct tcp_congestion_ops, release)) return &bpf_sk_setsockopt_proto; return NULL; @@ -177,7 +158,7 @@ bpf_tcp_ca_get_func_proto(enum bpf_func_id func_id, * The bpf-tcp-cc already has a more powerful way * to read tcp_sock from the PTR_TO_BTF_ID. */ - if (prog_ops_moff(prog) != + if (bpf_struct_ops_prog_moff(prog) != offsetof(struct tcp_congestion_ops, release)) return &bpf_sk_getsockopt_proto; return NULL; From patchwork Mon Feb 10 17:43:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968266 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C53982580CF; Mon, 10 Feb 2025 17:43:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209439; cv=none; b=EnvHhnt6f+X9IVFBb3OoidZrMZIdeykHiOj49BwV3iyp6qLTWEExfG2fp5OvS1FkVh4hoGLfM47gOZBrgpoU5fy1zc53cdjK9yo9I6zdFlPjOgiiP+bxqQXrUGeft+eW/6vgKdFTfkqtQNA2Otdt5AQBxHq68o5L9/yiw//NvkA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209439; c=relaxed/simple; bh=2/6mk8qHHpw2vFoIGo7pjNp/7fBVtYsbm2C3K0CJyEo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iAil4BWgnYm2K5evykrTw7vDk0kFWZSbTa6nvSN8l8ig0SID59mW+FPuqATBP7Pz2Oam+i/XXz55ZJ5dPzJq9YsdtwVH3pLFbSk08jnMdFGYapvFabnf/brHKarFHHkICWu2ni1KkO5E+SMM55M+skd8S00tiW+5xM6dihSucOI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Iev65ItV; arc=none smtp.client-ip=209.85.216.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Iev65ItV" Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-2f441791e40so6426793a91.3; Mon, 10 Feb 2025 09:43:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209437; x=1739814237; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nx/Qqdr7ZlpR07gYfhh2QpaDbLO/fjTlwySikXAdifs=; b=Iev65ItVgyo7JUoutQCWELGoYkIzt6Qrj3JlbXmADvYVmtkGIv+SpvzlxgQsx4h/yt 3mq8ZzqOswbS7+HSIMoP+v55tmTDOtUk+PNdjgJhmBi1XZlB9i2th0v4R0gPTUz962Ug n9RKcDAgadL3CLTFRv2uhUGyGQbmKYL0DW6YXTeP3Mriy+WOHK/UUPys5Vx3d9SkEh2A qvsjZhVWlPr227E1eC8p0B31wZidW5sGCwwJkdKdkSsEdWzyalwJBNAE/gMuXX2HpJ/5 ance2NzRyEIUlzHWFra2MP6OBH1KEbOQhoDAYjB7pUmc7gajLuFB7P6c1xc7/DuBfIiX jICw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209437; x=1739814237; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nx/Qqdr7ZlpR07gYfhh2QpaDbLO/fjTlwySikXAdifs=; b=dgyweW1oozDZCsa3zLoAzASwKNXcY813f5rg9KBO9KHJf1GEIsBcaTMRPZQU6+lwpF nZ7HC6yNEAz7X5Gy3smPt4XfWFxyDcapfe+WIK8ZR4j3hI8kR4LUZIloESKAWpIS0ddk 4OIaE0Ggc3nYy38owgWNMjLFJETiHWV2xHRqLKIlElhFm5yg46ebe/wesaTwOl/lPlou fcKtjfXPS9IlVdcz50DgQPU9/6Ejw8urUHMN0Vr62k2Mb4Nq9mQxBnoRKTFI6adOGE5A ghA9twEV4FFxl8tGhXAwY0wytyQNTMpZveh3OIjeHsaSxnX8x+peza/eRRlIxHNqQcNh H7KQ== X-Gm-Message-State: AOJu0YzNX/0bXegJCmsCjJmIBAIJvox71qUNM80P7EaARxPHx4ECK8S9 L30HoLgUbiNTYIGhgdRbBGEhraLn6wQ6diz+H/NYPYbIxVu2SLL8Tkfz3Wqe X-Gm-Gg: ASbGncuN7IyOtU5BqRoHLr9WIMAHrVg3OCt60k3aGJ5xwS2y0R6uAFuJK0OqDPyqfWB yFDlY5ttRQ0sTWNszMmzzh4Bxt28Z9Ts5DXdznqTkCqWWNpvbr6U9zjfkVEDm0TcqQVSL9nBBQX hMkAyVKWiZ08RX7k7w0moZkTUGkUIugFyt3g36Y6DmSNNxIiVUFzjYbDrAP7QJoegPZVi/oDyRd FYkeAm+Tj5cDCsRn6ZVnSTM66rriXhS84vvZWms252BdUKJqX31IxFeCgCd9Qf58m8/hUlu2EuF yMwfUgODWkIAAUiArxYuX93XAH0r+W8croJZdynnjsxYTcb8J5lkhu6Gx8SaBtR7eQ== X-Google-Smtp-Source: AGHT+IH+VGdX/HQE7ZgUxZN/Z390yj3CkkVyxbmbOdQdNjGAfFZdl+NpDhm6RgYlOxeTF41KuY9a+w== X-Received: by 2002:a17:90b:4c02:b0:2ee:a4f2:b311 with SMTP id 98e67ed59e1d1-2fa2406471emr22364237a91.8.1739209435275; Mon, 10 Feb 2025 09:43:55 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.43.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:43:54 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 08/19] bpf: net_sched: Support implementation of Qdisc_ops in bpf Date: Mon, 10 Feb 2025 09:43:22 -0800 Message-ID: <20250210174336.2024258-9-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Enable users to implement a classless qdisc using bpf. The last few patches in this series has prepared struct_ops to support core operators in Qdisc_ops. The recent advancement in bpf such as allocated objects, bpf list and bpf rbtree has also provided powerful and flexible building blocks to realize sophisticated scheduling algorithms. Therefore, in this patch, we start allowing qdisc to be implemented using bpf struct_ops. Users can implement Qdisc_ops.{enqueue, dequeue, init, reset, and .destroy in Qdisc_ops in bpf and register the qdisc dynamically into the kernel. Co-developed-by: Cong Wang Signed-off-by: Cong Wang Signed-off-by: Amery Hung Acked-by: Cong Wang --- net/sched/Kconfig | 12 +++ net/sched/Makefile | 1 + net/sched/bpf_qdisc.c | 210 ++++++++++++++++++++++++++++++++++++++++ net/sched/sch_api.c | 7 +- net/sched/sch_generic.c | 3 +- 5 files changed, 229 insertions(+), 4 deletions(-) create mode 100644 net/sched/bpf_qdisc.c diff --git a/net/sched/Kconfig b/net/sched/Kconfig index 8180d0c12fce..ccd0255da5a5 100644 --- a/net/sched/Kconfig +++ b/net/sched/Kconfig @@ -403,6 +403,18 @@ config NET_SCH_ETS If unsure, say N. +config NET_SCH_BPF + bool "BPF-based Qdisc" + depends on BPF_SYSCALL && BPF_JIT && DEBUG_INFO_BTF + help + This option allows BPF-based queueing disiplines. With BPF struct_ops, + users can implement supported operators in Qdisc_ops using BPF programs. + The queue holding skb can be built with BPF maps or graphs. + + Say Y here if you want to use BPF-based Qdisc. + + If unsure, say N. + menuconfig NET_SCH_DEFAULT bool "Allow override default queue discipline" help diff --git a/net/sched/Makefile b/net/sched/Makefile index 82c3f78ca486..904d784902d1 100644 --- a/net/sched/Makefile +++ b/net/sched/Makefile @@ -62,6 +62,7 @@ obj-$(CONFIG_NET_SCH_FQ_PIE) += sch_fq_pie.o obj-$(CONFIG_NET_SCH_CBS) += sch_cbs.o obj-$(CONFIG_NET_SCH_ETF) += sch_etf.o obj-$(CONFIG_NET_SCH_TAPRIO) += sch_taprio.o +obj-$(CONFIG_NET_SCH_BPF) += bpf_qdisc.o obj-$(CONFIG_NET_CLS_U32) += cls_u32.o obj-$(CONFIG_NET_CLS_ROUTE4) += cls_route.o diff --git a/net/sched/bpf_qdisc.c b/net/sched/bpf_qdisc.c new file mode 100644 index 000000000000..00f3232f4a98 --- /dev/null +++ b/net/sched/bpf_qdisc.c @@ -0,0 +1,210 @@ +#include +#include +#include +#include +#include +#include +#include + +static struct bpf_struct_ops bpf_Qdisc_ops; + +struct bpf_sk_buff_ptr { + struct sk_buff *skb; +}; + +static int bpf_qdisc_init(struct btf *btf) +{ + return 0; +} + +static const struct bpf_func_proto * +bpf_qdisc_get_func_proto(enum bpf_func_id func_id, + const struct bpf_prog *prog) +{ + /* Tail call is disabled since there is no gaurantee valid refcounted + * kptrs will always be passed to another bpf program with __ref arguments. + */ + switch (func_id) { + case BPF_FUNC_tail_call: + return NULL; + default: + return bpf_base_func_proto(func_id, prog); + } +} + +BTF_ID_LIST_SINGLE(bpf_sk_buff_ids, struct, sk_buff) +BTF_ID_LIST_SINGLE(bpf_sk_buff_ptr_ids, struct, bpf_sk_buff_ptr) + +static bool bpf_qdisc_is_valid_access(int off, int size, + enum bpf_access_type type, + const struct bpf_prog *prog, + struct bpf_insn_access_aux *info) +{ + struct btf *btf = prog->aux->attach_btf; + u32 arg; + + arg = btf_ctx_arg_idx(btf, prog->aux->attach_func_proto, off); + if (bpf_struct_ops_prog_moff(prog) == offsetof(struct Qdisc_ops, enqueue)) { + if (arg == 2 && type == BPF_READ) { + info->reg_type = PTR_TO_BTF_ID | PTR_TRUSTED; + info->btf = btf; + info->btf_id = bpf_sk_buff_ptr_ids[0]; + return true; + } + } + + return bpf_tracing_btf_ctx_access(off, size, type, prog, info); +} + +static int bpf_qdisc_btf_struct_access(struct bpf_verifier_log *log, + const struct bpf_reg_state *reg, + int off, int size) +{ + const struct btf_type *t, *skbt; + size_t end; + + skbt = btf_type_by_id(reg->btf, bpf_sk_buff_ids[0]); + t = btf_type_by_id(reg->btf, reg->btf_id); + if (t != skbt) { + bpf_log(log, "only read is supported\n"); + return -EACCES; + } + + switch (off) { + case offsetof(struct sk_buff, tstamp): + end = offsetofend(struct sk_buff, tstamp); + break; + case offsetof(struct sk_buff, priority): + end = offsetofend(struct sk_buff, priority); + break; + case offsetof(struct sk_buff, mark): + end = offsetofend(struct sk_buff, mark); + break; + case offsetof(struct sk_buff, queue_mapping): + end = offsetofend(struct sk_buff, queue_mapping); + break; + case offsetof(struct sk_buff, cb) + offsetof(struct qdisc_skb_cb, tc_classid): + end = offsetof(struct sk_buff, cb) + + offsetofend(struct qdisc_skb_cb, tc_classid); + break; + case offsetof(struct sk_buff, cb) + offsetof(struct qdisc_skb_cb, data[0]) ... + offsetof(struct sk_buff, cb) + offsetof(struct qdisc_skb_cb, + data[QDISC_CB_PRIV_LEN - 1]): + end = offsetof(struct sk_buff, cb) + + offsetofend(struct qdisc_skb_cb, data[QDISC_CB_PRIV_LEN - 1]); + break; + case offsetof(struct sk_buff, tc_index): + end = offsetofend(struct sk_buff, tc_index); + break; + default: + bpf_log(log, "no write support to sk_buff at off %d\n", off); + return -EACCES; + } + + if (off + size > end) { + bpf_log(log, + "write access at off %d with size %d beyond the member of sk_buff ended at %zu\n", + off, size, end); + return -EACCES; + } + + return 0; +} + +static const struct bpf_verifier_ops bpf_qdisc_verifier_ops = { + .get_func_proto = bpf_qdisc_get_func_proto, + .is_valid_access = bpf_qdisc_is_valid_access, + .btf_struct_access = bpf_qdisc_btf_struct_access, +}; + +static int bpf_qdisc_init_member(const struct btf_type *t, + const struct btf_member *member, + void *kdata, const void *udata) +{ + const struct Qdisc_ops *uqdisc_ops; + struct Qdisc_ops *qdisc_ops; + u32 moff; + + uqdisc_ops = (const struct Qdisc_ops *)udata; + qdisc_ops = (struct Qdisc_ops *)kdata; + + moff = __btf_member_bit_offset(t, member) / 8; + switch (moff) { + case offsetof(struct Qdisc_ops, peek): + qdisc_ops->peek = qdisc_peek_dequeued; + return 0; + case offsetof(struct Qdisc_ops, id): + if (bpf_obj_name_cpy(qdisc_ops->id, uqdisc_ops->id, + sizeof(qdisc_ops->id)) <= 0) + return -EINVAL; + return 1; + } + + return 0; +} + +static int bpf_qdisc_reg(void *kdata, struct bpf_link *link) +{ + return register_qdisc(kdata); +} + +static void bpf_qdisc_unreg(void *kdata, struct bpf_link *link) +{ + return unregister_qdisc(kdata); +} + +static int Qdisc_ops__enqueue(struct sk_buff *skb__ref, struct Qdisc *sch, + struct sk_buff **to_free) +{ + return 0; +} + +static struct sk_buff *Qdisc_ops__dequeue(struct Qdisc *sch) +{ + return NULL; +} + +static struct sk_buff *Qdisc_ops__peek(struct Qdisc *sch) +{ + return NULL; +} + +static int Qdisc_ops__init(struct Qdisc *sch, struct nlattr *arg, + struct netlink_ext_ack *extack) +{ + return 0; +} + +static void Qdisc_ops__reset(struct Qdisc *sch) +{ +} + +static void Qdisc_ops__destroy(struct Qdisc *sch) +{ +} + +static struct Qdisc_ops __bpf_ops_qdisc_ops = { + .enqueue = Qdisc_ops__enqueue, + .dequeue = Qdisc_ops__dequeue, + .peek = Qdisc_ops__peek, + .init = Qdisc_ops__init, + .reset = Qdisc_ops__reset, + .destroy = Qdisc_ops__destroy, +}; + +static struct bpf_struct_ops bpf_Qdisc_ops = { + .verifier_ops = &bpf_qdisc_verifier_ops, + .reg = bpf_qdisc_reg, + .unreg = bpf_qdisc_unreg, + .init_member = bpf_qdisc_init_member, + .init = bpf_qdisc_init, + .name = "Qdisc_ops", + .cfi_stubs = &__bpf_ops_qdisc_ops, + .owner = THIS_MODULE, +}; + +static int __init bpf_qdisc_kfunc_init(void) +{ + return register_bpf_struct_ops(&bpf_Qdisc_ops, Qdisc_ops); +} +late_initcall(bpf_qdisc_kfunc_init); diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c index e3e91cf867eb..1aad41b7d5a8 100644 --- a/net/sched/sch_api.c +++ b/net/sched/sch_api.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include @@ -358,7 +359,7 @@ static struct Qdisc_ops *qdisc_lookup_ops(struct nlattr *kind) read_lock(&qdisc_mod_lock); for (q = qdisc_base; q; q = q->next) { if (nla_strcmp(kind, q->id) == 0) { - if (!try_module_get(q->owner)) + if (!bpf_try_module_get(q, q->owner)) q = NULL; break; } @@ -1287,7 +1288,7 @@ static struct Qdisc *qdisc_create(struct net_device *dev, /* We will try again qdisc_lookup_ops, * so don't keep a reference. */ - module_put(ops->owner); + bpf_module_put(ops, ops->owner); err = -EAGAIN; goto err_out; } @@ -1398,7 +1399,7 @@ static struct Qdisc *qdisc_create(struct net_device *dev, netdev_put(dev, &sch->dev_tracker); qdisc_free(sch); err_out2: - module_put(ops->owner); + bpf_module_put(ops, ops->owner); err_out: *errp = err; return NULL; diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c index 14ab2f4c190a..e6fda9f20272 100644 --- a/net/sched/sch_generic.c +++ b/net/sched/sch_generic.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -1078,7 +1079,7 @@ static void __qdisc_destroy(struct Qdisc *qdisc) ops->destroy(qdisc); lockdep_unregister_key(&qdisc->root_lock_key); - module_put(ops->owner); + bpf_module_put(ops, ops->owner); netdev_put(dev, &qdisc->dev_tracker); trace_qdisc_destroy(qdisc); From patchwork Mon Feb 10 17:43:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968267 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5A768255E4B; Mon, 10 Feb 2025 17:43:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209439; cv=none; b=k/Koc36RCvz35oTTtBbYoA1ArDPTpksHOL2zn1XTfOsrCsXebSQxR865yW8NxbmoMIB+fjS27RBYKdwAeUlvHQQiQ9deIbgnH/bjF7PzEI+QT415+fDeL+LPEy/gG3v7mVqJrTMen04XQc0wGp7FwxAllZdZnHdDI1z65L0+cHs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209439; c=relaxed/simple; bh=RuzPRB79q27As0dh0HE5sZbuWOGTdkpmJkVJecz8KTo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=r8LCJ4AyYgpWYwRBWJWRJ89uOY07x75b4PCebkPSMOJlyxXL0NEqs6uwnrcfrvWwH+aNg8vkZFntHd3NOsmG50KflzVWynocNZSk+12w+HPCUGtNRnnNL0jkOWxK8WiJlFGL5SPs/XIHU3jgIDCUJd1wm134SDqeAJUiKYj2q1w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SffADZwH; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SffADZwH" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-21f74c4e586so38375075ad.0; Mon, 10 Feb 2025 09:43:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209437; x=1739814237; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WxfmY9SNILHvegwiEoTc4oD1Zqu0Ti6wlb2uVVZCDRI=; b=SffADZwH7RAoWCuiU6o24miU/6sttjZSiiq2ezOuu7hwl+lput8dXL32G5vIlPB6Zw Knwf1pTrRiPKfi9TYtrGtcSFk8hJ+BEEps8KnN7MBSX1kzkYZvIlkrjXT7k3/IlJ6wix XGhxGqxLXshEw1xyBPrUg/OkstKF4zN9Et23lDjJ56R7YWwLUJ5qZf6hoAsk4jTcYRkJ sLlPjF65bScWt1PJ8D74XANdQ/52wsoH2knHkyBUUVPF27UQA+jB62YGAQlo62gqNnvp OXuvArHqlJBAdrVvBFWvbpDzqNPapEM7FoTXV6vqBmV0mPYgbILXmC0IsLZ/FqPs+ew3 zziw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209437; x=1739814237; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WxfmY9SNILHvegwiEoTc4oD1Zqu0Ti6wlb2uVVZCDRI=; b=XGYeUxfY/OO4l4Y6Pzs8QjpKHT1pH9AvWndQcr3AhSBzWh6O+TC2zhg75++PY1n9yo Bd8jY7oM1LYeAYaPv0rpEYRVQI7OCz7vXdED49XJQVcfSdgj3XUyeKmqlcIiT1qDpktz t2SLSr7pTGVw5FoWqbonETd8fBeTPrIghTSPjXlU7coMNG3Vy4q0KAw9HTggBRBssx7l bGodCZZTs4vPYTMbMDXh0/ylEgsbRcO0OkUis5q6raazFZw30lkXswTn2BvH4B/QZ/4k A2MFJftVc5Qtnctx2C8bCalGT7kdPL/GxTZl4vFuQkcenKYHvP/H4Lj5K/FshH7rB9Xp B1cA== X-Gm-Message-State: AOJu0YymqFSS/kWLMAeHVSEXiTEcs7tu4V3ADlXRP2Q3qP9z5eKqQAtY 8vyePCX/pE5ybBeMy+bWySe+XpJhgpow9hsCmJdFD/1w1xw4OBVh+2No9KqQ X-Gm-Gg: ASbGncseD/tBfvKbyc+ACnJY5NUjsFD5i91xiyW7FqOgyenHp7Omv5wUBrMaKej+e4D VVtxBIg356TJUeKXIqYWoBnbjb1aflB8SKvtkxUpQrHnVgxYB9XFvoLp3Ogcl/9RKUua8CzPyW1 5mB4TJsAxWT4wvn4WCjpktTJEcYr6Oe40IAbwOJxP6wkmeG2ZNmOyaeZnigKdLDNU6I4KwM+bmA q0SkRPmKQ4Gws4vNa9PeKIKTZ0vJo2hOO3jtIU/p/xsrokLNwXalF36wHqLHJImQ68AwgDeLWc1 gDKBvMSQYBSvAdn094TCAhYyDK0uOuqO+mCoqsOQqBg9YCDEN3/J80R77TvEIey3/g== X-Google-Smtp-Source: AGHT+IEELJeYcLfqQVDuOdZPgKfl+WGDAcwyy+plD26K3rgg+owJS8Of2YwaRDUfcPiDv841hMv7mA== X-Received: by 2002:a17:90b:264a:b0:2fa:157e:c78e with SMTP id 98e67ed59e1d1-2fa9ed5d43fmr608648a91.7.1739209437371; Mon, 10 Feb 2025 09:43:57 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.43.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:43:56 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 09/19] bpf: net_sched: Add basic bpf qdisc kfuncs Date: Mon, 10 Feb 2025 09:43:23 -0800 Message-ID: <20250210174336.2024258-10-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Add basic kfuncs for working on skb in qdisc. Both bpf_qdisc_skb_drop() and bpf_kfree_skb() can be used to release a reference to an skb. However, bpf_qdisc_skb_drop() can only be called in .enqueue where a to_free skb list is available from kernel to defer the release. bpf_kfree_skb() should be used elsewhere. It is also used in bpf_obj_free_fields() when cleaning up skb in maps and collections. bpf_skb_get_hash() returns the flow hash of an skb, which can be used to build flow-based queueing algorithms. Finally, allow users to create read-only dynptr via bpf_dynptr_from_skb(). Signed-off-by: Amery Hung --- include/linux/bpf.h | 1 + kernel/bpf/bpf_struct_ops.c | 2 + net/sched/bpf_qdisc.c | 93 ++++++++++++++++++++++++++++++++++++- 3 files changed, 95 insertions(+), 1 deletion(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 6003ba36f6c5..bbca7b537cf8 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1810,6 +1810,7 @@ struct bpf_struct_ops { void *cfi_stubs; struct module *owner; const char *name; + const struct btf_type *type; struct btf_func_model func_models[BPF_STRUCT_OPS_MAX_NR_MEMBERS]; }; diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index d3a76f0c5a82..1ee6d41d4948 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -460,6 +460,8 @@ int bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, goto errout; } + st_ops->type = t; + return 0; errout: diff --git a/net/sched/bpf_qdisc.c b/net/sched/bpf_qdisc.c index 00f3232f4a98..69a1d547390c 100644 --- a/net/sched/bpf_qdisc.c +++ b/net/sched/bpf_qdisc.c @@ -111,6 +111,80 @@ static int bpf_qdisc_btf_struct_access(struct bpf_verifier_log *log, return 0; } +__bpf_kfunc_start_defs(); + +/* bpf_skb_get_hash - Get the flow hash of an skb. + * @skb: The skb to get the flow hash from. + */ +__bpf_kfunc u32 bpf_skb_get_hash(struct sk_buff *skb) +{ + return skb_get_hash(skb); +} + +/* bpf_kfree_skb - Release an skb's reference and drop it immediately. + * @skb: The skb whose reference to be released and dropped. + */ +__bpf_kfunc void bpf_kfree_skb(struct sk_buff *skb) +{ + kfree_skb(skb); +} + +/* bpf_qdisc_skb_drop - Drop an skb by adding it to a deferred free list. + * @skb: The skb whose reference to be released and dropped. + * @to_free_list: The list of skbs to be dropped. + */ +__bpf_kfunc void bpf_qdisc_skb_drop(struct sk_buff *skb, + struct bpf_sk_buff_ptr *to_free_list) +{ + __qdisc_drop(skb, (struct sk_buff **)to_free_list); +} + +__bpf_kfunc_end_defs(); + +BTF_KFUNCS_START(qdisc_kfunc_ids) +BTF_ID_FLAGS(func, bpf_skb_get_hash, KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_kfree_skb, KF_RELEASE) +BTF_ID_FLAGS(func, bpf_qdisc_skb_drop, KF_RELEASE) +BTF_ID_FLAGS(func, bpf_dynptr_from_skb, KF_TRUSTED_ARGS) +BTF_KFUNCS_END(qdisc_kfunc_ids) + +BTF_SET_START(qdisc_common_kfunc_set) +BTF_ID(func, bpf_skb_get_hash) +BTF_ID(func, bpf_kfree_skb) +BTF_ID(func, bpf_dynptr_from_skb) +BTF_SET_END(qdisc_common_kfunc_set) + +BTF_SET_START(qdisc_enqueue_kfunc_set) +BTF_ID(func, bpf_qdisc_skb_drop) +BTF_SET_END(qdisc_enqueue_kfunc_set) + +static int bpf_qdisc_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id) +{ + if (bpf_Qdisc_ops.type != btf_type_by_id(prog->aux->attach_btf, + prog->aux->attach_btf_id)) + return 0; + + /* Skip the check when prog->attach_func_name is not yet available + * during check_cfg(). + */ + if (!btf_id_set8_contains(&qdisc_kfunc_ids, kfunc_id) || + !prog->aux->attach_func_name) + return 0; + + if (bpf_struct_ops_prog_moff(prog) == offsetof(struct Qdisc_ops, enqueue)) { + if (btf_id_set_contains(&qdisc_enqueue_kfunc_set, kfunc_id)) + return 0; + } + + return btf_id_set_contains(&qdisc_common_kfunc_set, kfunc_id) ? 0 : -EACCES; +} + +static const struct btf_kfunc_id_set bpf_qdisc_kfunc_set = { + .owner = THIS_MODULE, + .set = &qdisc_kfunc_ids, + .filter = bpf_qdisc_kfunc_filter, +}; + static const struct bpf_verifier_ops bpf_qdisc_verifier_ops = { .get_func_proto = bpf_qdisc_get_func_proto, .is_valid_access = bpf_qdisc_is_valid_access, @@ -203,8 +277,25 @@ static struct bpf_struct_ops bpf_Qdisc_ops = { .owner = THIS_MODULE, }; +BTF_ID_LIST(bpf_sk_buff_dtor_ids) +BTF_ID(func, bpf_kfree_skb) + static int __init bpf_qdisc_kfunc_init(void) { - return register_bpf_struct_ops(&bpf_Qdisc_ops, Qdisc_ops); + int ret; + const struct btf_id_dtor_kfunc skb_kfunc_dtors[] = { + { + .btf_id = bpf_sk_buff_ids[0], + .kfunc_btf_id = bpf_sk_buff_dtor_ids[0] + }, + }; + + ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, &bpf_qdisc_kfunc_set); + ret = ret ?: register_btf_id_dtor_kfuncs(skb_kfunc_dtors, + ARRAY_SIZE(skb_kfunc_dtors), + THIS_MODULE); + ret = ret ?: register_bpf_struct_ops(&bpf_Qdisc_ops, Qdisc_ops); + + return ret; } late_initcall(bpf_qdisc_kfunc_init); From patchwork Mon Feb 10 17:43:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968268 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 743D0204863; Mon, 10 Feb 2025 17:43:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209440; cv=none; b=sDMb9QxPXt6LfHz22bRCjwuWTvArdI+rQj+vZqZpDMy++MS8RT3vx9KmHk9ZIR4Mx5h/HbKx8O1fD5cpz42tWvrk/fXblRhBcU6nF2SYzMgI6xQsTc6LnHXK9RmZlYHxfB11OO9eO4rSlnQeGk5OfpMNPpHqj6DePJfwMIJHBT0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209440; c=relaxed/simple; bh=aldpEaDnt4oDAJv5zeFxR3usl+NBly81q17OHUipcvY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KxVn3q68qPkF2VVG3r3Cu75u5SYu60EHfrO7o8dtfr1JmT/x/O4dLqVimFuDe4klrpzt+k6LGW3EJJMj757y3FEVZCy/IMBpuPN72iwUbhK/1096RtD1ZtHjRBYPM5jJE6hGdb2SU2kuIAlKT5bRgC7/v33sANjm6+itRZBbBdw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=jfkogBj7; arc=none smtp.client-ip=209.85.216.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jfkogBj7" Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-2fa1c093d6eso5353747a91.0; Mon, 10 Feb 2025 09:43:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209439; x=1739814239; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1Tb9zgB4Ux2lEBx5nRCfRcA5Fu7KXgIIpYuPhggdRh8=; b=jfkogBj7umAWtLRFGs7T+2ewHbECdwpyGTdJdybn96fOYWQdu0e4bAhF+NYsq1jAS9 2aGTQYt1L9kiQ6lQ116C2cr5gVWvr3zakWpzpDDBgXYOPsWs1uRMTcPGofrf2NTseuH5 BN6ABjB+CYfGCDR5IPgTtb7XzocD254OgASWwIHxU3dZtHWf9xY38B17a6Ov703rKWS7 SMJbFjW3RHs08gmypl8R/HExm0Bryu9j9x5NBINT/5xwmblj/ETVm3zCYI2S1/HOJFCP o9HTALCaNDoqFxYmuO1IQ3Le3GyfY4YlHdl2mcvXQGmWJRZUBJ9DEflikGT6sofU7dUB L6+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209439; x=1739814239; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1Tb9zgB4Ux2lEBx5nRCfRcA5Fu7KXgIIpYuPhggdRh8=; b=j9nsHsrJyf1Nz0fepefHuy11s6JqIDPUqjXsFWWNwf+RZAwX6doD0pEnfy8TE1L6cC HliJEM91p+FQIH6f2FScWrUqR4vhQ9FT58J2O0XjCPpnzxGdy3kLUYFSNCgz86LhhxxI OUTfXSEVoco8qPVKkJkYq6HZVbA1TkuvK0CkTLExxeG4GqcmN/87jsAPnw/pqWtArpKh yUiS3sG+MjzFqMyhjp1kdE2Eqyo9HdknhPXJpMlHWbAqGk/uIDVAYhofkpjhaFTMY/5w 1Rq4FUIVAI/suYnkNWG+RWPu8DAeBbGz399Fjnd/+obueykBJLsc8d3VLoKSz7kQibnH tUkw== X-Gm-Message-State: AOJu0YxRPjdb+CJWVAc4R/x9gHgPTQCO7TI80JE0/3prE+5Lj5aONf2n 47AwulumchsXy4zjVbr2fVYFZsS1oaAzpKB7TnZla8MAES5I4bqUJKIIm699 X-Gm-Gg: ASbGncsxpEeFf8J24O6M9P9fzKEH48VxkSIOLqvYouv/BZWu0eoHyMgZkJzsukWdSVN YZR/KlG/878ja7dim3PO3K3WF5TVub2SFrtDhmSKkxRrggPNH0+zuyEBKzCzXmwLWzRgp0MbQr7 QA0eZSIdr8ZR8GL/gcnjlX3DlnZSTOV+vZNrI1FGnpLPGMMmtg9CTrvrYuzLQL8AddabuZjK4ty Rw7wDpKAe02CoZBngAU44LtwX/oCXpW9axUujc+0esBVFuy3PKIeEffwSDTSF+Q/P8qRYgw7sdl hp2UlelXJ34t6mJhAbaGhZKTKTzHbJ+TSuAjET8A5S82z73vXdt/pLlFqhwewb3uLQ== X-Google-Smtp-Source: AGHT+IH3KUOlK5GB0ml4pOfR5WCxpQyWQoJ0PaamE/DnQp5aZFwF84EEuN7v3xLPIEmWWIQy5v7WMA== X-Received: by 2002:a17:90b:3dc3:b0:2fa:1451:2d56 with SMTP id 98e67ed59e1d1-2fa243e10b6mr22378988a91.25.1739209438669; Mon, 10 Feb 2025 09:43:58 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.43.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:43:58 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 10/19] bpf: Search and add kfuncs in struct_ops prologue and epilogue Date: Mon, 10 Feb 2025 09:43:24 -0800 Message-ID: <20250210174336.2024258-11-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Currently, add_kfunc_call() is only invoked once before the main verification loop. Therefore, the verifier could not find the bpf_kfunc_btf_tab of a new kfunc call which is not seen in user defined struct_ops operators but introduced in gen_prologue or gen_epilogue during do_misc_fixup(). Fix this by searching kfuncs in the patching instruction buffer and add them to prog->aux->kfunc_tab. Signed-off-by: Amery Hung Acked-by: Eduard Zingerman --- kernel/bpf/verifier.c | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5bcf095e8d0c..c11d105b3c6f 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -3215,6 +3215,21 @@ bpf_jit_find_kfunc_model(const struct bpf_prog *prog, return res ? &res->func_model : NULL; } +static int add_kfunc_in_insns(struct bpf_verifier_env *env, + struct bpf_insn *insn, int cnt) +{ + int i, ret; + + for (i = 0; i < cnt; i++, insn++) { + if (bpf_pseudo_kfunc_call(insn)) { + ret = add_kfunc_call(env, insn->imm, insn->off); + if (ret < 0) + return ret; + } + } + return 0; +} + static int add_subprog_and_kfunc(struct bpf_verifier_env *env) { struct bpf_subprog_info *subprog = env->subprog_info; @@ -20368,7 +20383,7 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env) { struct bpf_subprog_info *subprogs = env->subprog_info; const struct bpf_verifier_ops *ops = env->ops; - int i, cnt, size, ctx_field_size, delta = 0, epilogue_cnt = 0; + int i, cnt, size, ctx_field_size, ret, delta = 0, epilogue_cnt = 0; const int insn_cnt = env->prog->len; struct bpf_insn *epilogue_buf = env->epilogue_buf; struct bpf_insn *insn_buf = env->insn_buf; @@ -20397,6 +20412,10 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env) return -ENOMEM; env->prog = new_prog; delta += cnt - 1; + + ret = add_kfunc_in_insns(env, epilogue_buf, epilogue_cnt - 1); + if (ret < 0) + return ret; } } @@ -20417,6 +20436,10 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env) env->prog = new_prog; delta += cnt - 1; + + ret = add_kfunc_in_insns(env, insn_buf, cnt - 1); + if (ret < 0) + return ret; } } From patchwork Mon Feb 10 17:43:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968269 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C29D22586DC; Mon, 10 Feb 2025 17:44:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209442; cv=none; b=ntbpxnX2Ig+bgetdjapdoosB8p80BjXwXVvC0xM3Vr5jaMd1erAZuxrW/ZlOCLZhXeQ10CTOPAWHzGGZNldWWQ9tq++0e+/UUgk5jz3DGr72L1B4iHxTyNiEkBClz8viDLNwn25nG7pH+WXt4SyfkhaejOThJaHXsYFLhf/BGpo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209442; c=relaxed/simple; bh=Q8ePHSEjKZcDqGD7K2WW4ZeTYIIw7I437z5FVcysuA0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GuthaeBZAxBNENmGPEXdctPytqIlJ7VXpNTZm+VgWCRmTj15zITbOy+PR8JfJKBoER5cxOA1WYcj7DvYNM2GZ6maSmsuUkkPa1tY/hJ+lXZ2OUXI4y8m/UaWELQtzbVZ5VWuwAtXFaED7qa/D5AeQzx/w2rS0Yguh4zLHtgIfUw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IhlqqTUe; arc=none smtp.client-ip=209.85.216.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IhlqqTUe" Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-2fa2c1ab145so5808627a91.3; Mon, 10 Feb 2025 09:44:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209440; x=1739814240; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SgHDOpSEEIi90UDqwqeFyvDF2tZ+G1vUb9y1obneiyQ=; b=IhlqqTUes3nzhbm/vcCr6Z/mVIc4KjmjaMdUf7ha74EFuY7uUBs/1mBhHyYMVSEgJL wEGdhc+QiTyn6s89fhRQ3Bnxi3Cu5t+FA2+bWzS4qS3uBD0YNiIYgnQH599psmezpwHx Z9/GSkjPYz16n0IJnU8A8ybgRgXUQJzCchCGwFR3VSd9CNpGkrgyuiymUKg3Qvud7/Rn OyGYlZwEcvWCGIdercSeULeNKaMY1qafOc0ROwJ3HPf+8W4Ie62Zd+CHvTmAe96+L2I6 R8CNqoKQqbbzSqyLG6cb2OTA6rwDa2rufEwxcFlxdNjwnvmSbVDY64LUT7mBC/TKjc5H FpQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209440; x=1739814240; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SgHDOpSEEIi90UDqwqeFyvDF2tZ+G1vUb9y1obneiyQ=; b=uDutkeaZnOm2enJMqd2SRK/LTtG03nJpljsxrDf6u3oBotsCXZZI9K+L6Lt6MhsfPJ tugz1vG+ZGROUTVocEiOYtiNMyVBV+vVbe0ag1JRZ1NXxJe+x2IVVbCrcWwHUg+ZffJS 5e5YMXeGTYxlYzgcMiMYqOz27VgA7hLwOJvqQuIxkOrK36Vcu2srzntKdx1Vi+WtfpaK U16wGxY7Cri5/K9Qu6aoPJghaoG4pJ58/2uQagxtCYGOZS5tuYN3abKDL66JaQlPo3yJ b0bM2Uc9wKbW2tE5QLI4Mht3dNkVcbuDopU9L6TPOF0UVj84SkUtvbpcuB6LXln/MGkX rcOg== X-Gm-Message-State: AOJu0Yzi0KtGYf8cfarJXgGVGWOY2Um17yKDrpUPi4/Oox5mNsGDuxS3 HwP/rN5mlitnegDeLCNj6ZxRgMVio+/iOLu7mQteVkR88Q5c7khdNAyHbDxJ X-Gm-Gg: ASbGncvFTWW/18ANaDt7g2wm4ajp91/p4u9G8aaQcGsfXP3Rlo6R/gXs0KvM7/aMr4w W5Xp74dzwT2hPpG/4fgg1ngxS0Zs0qI/s0qAJE1MilaJVkdkVYh6a3m8oaYFxU6if+celyT85rr bAjv3PQSdXTRhHgaPsdqVbKvzNzMUy1s6n3/TLT8C1GiZ/weHjQjBzb1kwDCM88NX7a6CA0F/1R It0KAWcdP9FbgUi7syK+pCMAAXYc8dLAyoIwM5XJnT5+SHNWNAulLKS8CRz2xErJ2L/Jn+W+BZG 1MQUAzySUp8B7iYONb1DlfFJGoQWPZGVTTBQxtb4KpqulKHc5FtkgkSU5Utq8O0LJw== X-Google-Smtp-Source: AGHT+IEQqnxPXS1c74ypTB+MLo81V7cJ0DBoV77yo2/D7QFCDZhEctHwmyqUyPG9lTBkqz7W8MqV6A== X-Received: by 2002:a17:90b:3d03:b0:2ea:bf1c:1e3a with SMTP id 98e67ed59e1d1-2fa24069ec6mr27233219a91.12.1739209439842; Mon, 10 Feb 2025 09:43:59 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.43.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:43:59 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 11/19] bpf: net_sched: Add a qdisc watchdog timer Date: Mon, 10 Feb 2025 09:43:25 -0800 Message-ID: <20250210174336.2024258-12-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Add a watchdog timer to bpf qdisc. The watchdog can be used to schedule the execution of qdisc through kfunc, bpf_qdisc_schedule(). It can be useful for building traffic shaping scheduling algorithm, where the time the next packet will be dequeued is known. Signed-off-by: Amery Hung --- include/linux/filter.h | 10 +++++ net/sched/bpf_qdisc.c | 92 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 102 insertions(+) diff --git a/include/linux/filter.h b/include/linux/filter.h index a3ea46281595..3ed6eb9e7c73 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -469,6 +469,16 @@ static inline bool insn_is_cast_user(const struct bpf_insn *insn) .off = 0, \ .imm = BPF_CALL_IMM(FUNC) }) +/* Kfunc call */ + +#define BPF_CALL_KFUNC(OFF, IMM) \ + ((struct bpf_insn) { \ + .code = BPF_JMP | BPF_CALL, \ + .dst_reg = 0, \ + .src_reg = BPF_PSEUDO_KFUNC_CALL, \ + .off = OFF, \ + .imm = IMM }) + /* Raw code statement block */ #define BPF_RAW_INSN(CODE, DST, SRC, OFF, IMM) \ diff --git a/net/sched/bpf_qdisc.c b/net/sched/bpf_qdisc.c index 69a1d547390c..ae06637f4bab 100644 --- a/net/sched/bpf_qdisc.c +++ b/net/sched/bpf_qdisc.c @@ -8,6 +8,10 @@ static struct bpf_struct_ops bpf_Qdisc_ops; +struct bpf_sched_data { + struct qdisc_watchdog watchdog; +}; + struct bpf_sk_buff_ptr { struct sk_buff *skb; }; @@ -111,6 +115,46 @@ static int bpf_qdisc_btf_struct_access(struct bpf_verifier_log *log, return 0; } +BTF_ID_LIST(bpf_qdisc_init_prologue_ids) +BTF_ID(func, bpf_qdisc_init_prologue) + +static int bpf_qdisc_gen_prologue(struct bpf_insn *insn_buf, bool direct_write, + const struct bpf_prog *prog) +{ + struct bpf_insn *insn = insn_buf; + + if (bpf_struct_ops_prog_moff(prog) != offsetof(struct Qdisc_ops, init)) + return 0; + + *insn++ = BPF_MOV64_REG(BPF_REG_6, BPF_REG_1); + *insn++ = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0); + *insn++ = BPF_CALL_KFUNC(0, bpf_qdisc_init_prologue_ids[0]); + *insn++ = BPF_MOV64_REG(BPF_REG_1, BPF_REG_6); + *insn++ = prog->insnsi[0]; + + return insn - insn_buf; +} + +BTF_ID_LIST(bpf_qdisc_reset_destroy_epilogue_ids) +BTF_ID(func, bpf_qdisc_reset_destroy_epilogue) + +static int bpf_qdisc_gen_epilogue(struct bpf_insn *insn_buf, const struct bpf_prog *prog, + s16 ctx_stack_off) +{ + struct bpf_insn *insn = insn_buf; + + if (bpf_struct_ops_prog_moff(prog) != offsetof(struct Qdisc_ops, reset) && + bpf_struct_ops_prog_moff(prog) != offsetof(struct Qdisc_ops, destroy)) + return 0; + + *insn++ = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_FP, ctx_stack_off); + *insn++ = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0); + *insn++ = BPF_CALL_KFUNC(0, bpf_qdisc_reset_destroy_epilogue_ids[0]); + *insn++ = BPF_EXIT_INSN(); + + return insn - insn_buf; +} + __bpf_kfunc_start_defs(); /* bpf_skb_get_hash - Get the flow hash of an skb. @@ -139,6 +183,36 @@ __bpf_kfunc void bpf_qdisc_skb_drop(struct sk_buff *skb, __qdisc_drop(skb, (struct sk_buff **)to_free_list); } +/* bpf_qdisc_watchdog_schedule - Schedule a qdisc to a later time using a timer. + * @sch: The qdisc to be scheduled. + * @expire: The expiry time of the timer. + * @delta_ns: The slack range of the timer. + */ +__bpf_kfunc void bpf_qdisc_watchdog_schedule(struct Qdisc *sch, u64 expire, u64 delta_ns) +{ + struct bpf_sched_data *q = qdisc_priv(sch); + + qdisc_watchdog_schedule_range_ns(&q->watchdog, expire, delta_ns); +} + +/* bpf_qdisc_init_prologue - Hidden kfunc called in prologue of .init. */ +__bpf_kfunc void bpf_qdisc_init_prologue(struct Qdisc *sch) +{ + struct bpf_sched_data *q = qdisc_priv(sch); + + qdisc_watchdog_init(&q->watchdog, sch); +} + +/* bpf_qdisc_reset_destroy_epilogue - Hidden kfunc called in epilogue of .reset + * and .destroy + */ +__bpf_kfunc void bpf_qdisc_reset_destroy_epilogue(struct Qdisc *sch) +{ + struct bpf_sched_data *q = qdisc_priv(sch); + + qdisc_watchdog_cancel(&q->watchdog); +} + __bpf_kfunc_end_defs(); BTF_KFUNCS_START(qdisc_kfunc_ids) @@ -146,6 +220,9 @@ BTF_ID_FLAGS(func, bpf_skb_get_hash, KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_kfree_skb, KF_RELEASE) BTF_ID_FLAGS(func, bpf_qdisc_skb_drop, KF_RELEASE) BTF_ID_FLAGS(func, bpf_dynptr_from_skb, KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_qdisc_watchdog_schedule, KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_qdisc_init_prologue, KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_qdisc_reset_destroy_epilogue, KF_TRUSTED_ARGS) BTF_KFUNCS_END(qdisc_kfunc_ids) BTF_SET_START(qdisc_common_kfunc_set) @@ -156,8 +233,13 @@ BTF_SET_END(qdisc_common_kfunc_set) BTF_SET_START(qdisc_enqueue_kfunc_set) BTF_ID(func, bpf_qdisc_skb_drop) +BTF_ID(func, bpf_qdisc_watchdog_schedule) BTF_SET_END(qdisc_enqueue_kfunc_set) +BTF_SET_START(qdisc_dequeue_kfunc_set) +BTF_ID(func, bpf_qdisc_watchdog_schedule) +BTF_SET_END(qdisc_dequeue_kfunc_set) + static int bpf_qdisc_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id) { if (bpf_Qdisc_ops.type != btf_type_by_id(prog->aux->attach_btf, @@ -174,6 +256,9 @@ static int bpf_qdisc_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id) if (bpf_struct_ops_prog_moff(prog) == offsetof(struct Qdisc_ops, enqueue)) { if (btf_id_set_contains(&qdisc_enqueue_kfunc_set, kfunc_id)) return 0; + } else if (bpf_struct_ops_prog_moff(prog) == offsetof(struct Qdisc_ops, dequeue)) { + if (btf_id_set_contains(&qdisc_dequeue_kfunc_set, kfunc_id)) + return 0; } return btf_id_set_contains(&qdisc_common_kfunc_set, kfunc_id) ? 0 : -EACCES; @@ -189,6 +274,8 @@ static const struct bpf_verifier_ops bpf_qdisc_verifier_ops = { .get_func_proto = bpf_qdisc_get_func_proto, .is_valid_access = bpf_qdisc_is_valid_access, .btf_struct_access = bpf_qdisc_btf_struct_access, + .gen_prologue = bpf_qdisc_gen_prologue, + .gen_epilogue = bpf_qdisc_gen_epilogue, }; static int bpf_qdisc_init_member(const struct btf_type *t, @@ -204,6 +291,11 @@ static int bpf_qdisc_init_member(const struct btf_type *t, moff = __btf_member_bit_offset(t, member) / 8; switch (moff) { + case offsetof(struct Qdisc_ops, priv_size): + if (uqdisc_ops->priv_size) + return -EINVAL; + qdisc_ops->priv_size = sizeof(struct bpf_sched_data); + return 1; case offsetof(struct Qdisc_ops, peek): qdisc_ops->peek = qdisc_peek_dequeued; return 0; From patchwork Mon Feb 10 17:43:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968270 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B791F2586F3; Mon, 10 Feb 2025 17:44:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209443; cv=none; b=l8d2b+X+X4DzgeaSWCFJv7/y8czG2DovCVyU5FiZ7ZJR0muzhnmoG5hxOGjaPw5JxYLK5dUHf9FEwCxb4KS0Vhq1tTkUjbjh5RK8R2CYw9XVVtL3XbfM2KEC/jhYR7nD1LiJv7bgeoZuVAmA5/FcLghF282CDR54dmISCOD+76A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209443; c=relaxed/simple; bh=PvsjZwquHBtkiD0ex5OqAaIb81/a5+rvsxROvzoxNf4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LLyuxYORHms2A+jYNchZ6IxWVaJG6aNaSknvBN1DRWPohM9b+DnfIOSoCUep+mMOYjHRtBtjfWkoxRTpwnlA+qY0d2grWMWVciaA6Rz7tZf3I4LXhzIuIYwSI9W/IeIQUTbcf3HJi2mVTMGQ0kKE4SE0Uhqf1f2l8Vdg1Jshd88= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=PSTlrHXv; arc=none smtp.client-ip=209.85.216.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PSTlrHXv" Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2fa7465baceso2518728a91.0; Mon, 10 Feb 2025 09:44:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209441; x=1739814241; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Jp4xQH7m5lEMdSS67IiNyx4buTe0VVGMGzphEL5Q8YM=; b=PSTlrHXvcMz/sCbM9cjQUV6IuoKgxUG28IoTPUXtHxcX8Gx4x8OFXQ1Te0gw13yXHe DKqFthpl538C2P6AtiWEhWPkBwMUgzIHz/na8h2PyvNo/FVfwbNfbkfE+rSTqRwS5jHF 2CMWZoWmFl/6c1HAdteu6XPnY2d+fb8nxIFyP9wdkUNma6JuKizRRA3l4VOxmri2octP GdVYMzotbqrmwtLvz2Kzf+UuyVDRwOiP1K0ZRJE5OuvqOnHG47e7bQbxvXMRO0K02+Vf P5m6lAOGRuJgh3RrJ4hlfP3qVxmexmV+aJRpfMYD1BqZ9FP7NVOmlfVkkL890woD8N1i LxSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209441; x=1739814241; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jp4xQH7m5lEMdSS67IiNyx4buTe0VVGMGzphEL5Q8YM=; b=FBT7rNbzgkPk5JS5LPEui/4hWQa/jbgednmqT4a2ZHsFvF+w2U0r3+0tAZioEYAcJV SAwxXxIYzMJK+hSNORIQ5Qy6S1d+3nzGx6V6dOPGNwasZpYupX9FHVq6cwo2yT+kQdl/ RVQ6ggd4jNOT8b3EK8nE1xJaVGdSSXxRkmqtGQfK33T5q9Jp1zvRzkPNDstFwCDNcTyr BWRHdZyDWl+oJ75D62R2nU7HiTEaKYlMFs1euRqOOpe8nVsJSc6fIwtw4aZSHzoq5+T/ lfRzJQ/78Wb+vDTJD8FHZkbIskPuZWYJCfNTiv5aeA64xG6WWuk6kxh6KPlms2uA/QT/ Vpjw== X-Gm-Message-State: AOJu0YxmtJq3EeQpFaN7g2rigaiuxSc/lP3/9OxX0tKPgg0nuA+5f4AU AdFQ0SNwdgIMMu4gJ2+MLPk4mKKkNlNFBi80OuIZCmcHqie+ggT1Rm1r3Ldo X-Gm-Gg: ASbGncviGctB6WomN1ZGiZq35fRmDdnKWlWrlx5iGY+UzPTL/H0mT75d88FeRNCxWHC JMh2s9MPRA+na8zc0GtIQvb/MUb/5m8osPrr04IxS+3zqDcE0KwN+NYpnBHILno76A1sbGWBIYT bvTr5TiWvu8bxVjjAAJrM7yUs7hhTYQdCq62/NwygoanneE06VM4K+5rcn41/ADLoqrtBorTw21 Eltd+WoruGIHLpT05uc7gRwrVnfb0DkpP+WLT1OoThZ23N5RGML03lwtJudkDLEhdl0rzV/+0Mk pBqtz2Fmjk9dTJjGLdzrnFOOoCM7Xe5ZK8l1LUd35HYExwqeRj0uKUPfeGoqc68W1g== X-Google-Smtp-Source: AGHT+IHvNGQ+ztuqQK9ZnlrTb3Mcfg+sxnBLcJHC6K/i/pTxHP0oJdvSdgxyT8WSossa/YWQIw2g9w== X-Received: by 2002:a17:90b:38c3:b0:2f4:434d:c7ed with SMTP id 98e67ed59e1d1-2fa24177361mr24714444a91.16.1739209440932; Mon, 10 Feb 2025 09:44:00 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.44.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:44:00 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 12/19] bpf: net_sched: Support updating bstats Date: Mon, 10 Feb 2025 09:43:26 -0800 Message-ID: <20250210174336.2024258-13-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Add a kfunc to update Qdisc bstats when an skb is dequeued. The kfunc is only available in .dequeue programs. Signed-off-by: Amery Hung --- net/sched/bpf_qdisc.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/net/sched/bpf_qdisc.c b/net/sched/bpf_qdisc.c index ae06637f4bab..edf01f3f1c2a 100644 --- a/net/sched/bpf_qdisc.c +++ b/net/sched/bpf_qdisc.c @@ -213,6 +213,15 @@ __bpf_kfunc void bpf_qdisc_reset_destroy_epilogue(struct Qdisc *sch) qdisc_watchdog_cancel(&q->watchdog); } +/* bpf_qdisc_bstats_update - Update Qdisc basic statistics + * @sch: The qdisc from which an skb is dequeued. + * @skb: The skb to be dequeued. + */ +__bpf_kfunc void bpf_qdisc_bstats_update(struct Qdisc *sch, const struct sk_buff *skb) +{ + bstats_update(&sch->bstats, skb); +} + __bpf_kfunc_end_defs(); BTF_KFUNCS_START(qdisc_kfunc_ids) @@ -223,6 +232,7 @@ BTF_ID_FLAGS(func, bpf_dynptr_from_skb, KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_qdisc_watchdog_schedule, KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_qdisc_init_prologue, KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_qdisc_reset_destroy_epilogue, KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_qdisc_bstats_update, KF_TRUSTED_ARGS) BTF_KFUNCS_END(qdisc_kfunc_ids) BTF_SET_START(qdisc_common_kfunc_set) @@ -238,6 +248,7 @@ BTF_SET_END(qdisc_enqueue_kfunc_set) BTF_SET_START(qdisc_dequeue_kfunc_set) BTF_ID(func, bpf_qdisc_watchdog_schedule) +BTF_ID(func, bpf_qdisc_bstats_update) BTF_SET_END(qdisc_dequeue_kfunc_set) static int bpf_qdisc_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id) From patchwork Mon Feb 10 17:43:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968271 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D0D0E259486; Mon, 10 Feb 2025 17:44:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209444; cv=none; b=Kcg5CnrYdoMoAjEoeLu+fERQhZ8g3738la64zud0mKgqejpcZFuDq7j0VYUs4zAu6KSMuD9zSDQB5nrk0E2kQg6OGTSBRav8xzCrdiQ8C7gFPiDN+Ft5wgjGxd/49Dffiun2HVgVzYRsf9s6W2s1NCII6DPtUgmBg0VBZJpG5ck= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209444; c=relaxed/simple; bh=vPF4pu/P8uuuCviRPth91G084ytUrBykEU3WXlDjZ8U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=po/+LmhU+9eO8phS0/Ra3sKjJKpZ7DkjU0MiRbw2PTPYWR1jEM7i2XWiJOaByo9y3jQ/0A333Qt30O7Y+CidsJEA+LYov8+I3A15ul68ycgdoM8clhjKZ5ix0e8CetmUvg9tqvqxNYDamg6K/r/mhYhxr3faD15zp5221ev2sL4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=a0XIylxk; arc=none smtp.client-ip=209.85.216.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="a0XIylxk" Received: by mail-pj1-f41.google.com with SMTP id 98e67ed59e1d1-2fa2c456816so4659367a91.1; Mon, 10 Feb 2025 09:44:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209442; x=1739814242; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eY+Y9OxGeDSUuz0ekM5BZDl8hgTH17GlThABvm9XWiw=; b=a0XIylxkEEbGMxqYCjDpObWFWFS4CVhJHd76ajp4DDOJasEHFnd99au5shqceVdKnx XUQqQ6N8/J4DDNGm0QV8H0aRkmyPaPn9UiuAjFwrTQu33zy1ZQ9i8t6yVg0D3LMWaQbd yPFVFBiFzQ73znbKafxcx0oOuj/tp5densJHIyEtHrTr7PFnvGN12u7sUHoZe91803cN +7gIPAxTB8KQhE1t8cye/+gDBnWKkBiTbLPDGcgJhY1xUPiYIuvrqWbc9az6GMx6MVb8 U6XrbCHcNuPW91DnWPtUcSVSuDYCWJTqeAsJx9gTNSFvyIdhhu80PUmJjhVYf5DCJCuL +E0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209442; x=1739814242; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eY+Y9OxGeDSUuz0ekM5BZDl8hgTH17GlThABvm9XWiw=; b=wfNRMgyJd+RnWYyL656KeNtOMRwnfkpo4/du+jbXAkx4LW2L4q2MlEYqez3+uHNWMR lRMRFGBESI2tLFkQuZAWhnVWBUcD9pF0p61R5bIuTknobPVIP//eh4b+kt2yQ+EsKTSe kCAHOImcF9acyNcL+gi4xu3RtOPF3KET5DCckwwyw0Iv1AvoVcKEGOKeWz3XaWDJpXHd 9y/MxnR8tLk8fESHaznzWw+7M+z3FKup+M/Zpm5PBpDwCCdhCTz7+64S8KBTJb8h4llV /E3ZRLnMupgUDpGMLLAsPAQc00CgdqFGhLZFLyvsY8NwWOoDI5wCuFDQHQxKX/cyNg4q OR4A== X-Gm-Message-State: AOJu0YxmtVn+k76Uel6dTb00kVi96QzVEnYqF+cgx5OpRgNw/t7pjnqU JZq/NVw/4YaaDjFdMA1MLXGu9bF47NKmVtid2DYhOctpokS2W/VIumOeG+iH X-Gm-Gg: ASbGncvGdzW+fT6K1zUg8+zxrkCuQh1XcuT7+1fx9SHR2Cr7Ex4XKBXU/OFU/iNpKbH bY6j5auDgYVrz5GQnQoVh9kNvNHe50HFyDfwyeEqClxi9XdrPQLoKG2u3uQ/rGdBIeIDyeH3RSx AhTclInRkk8XbRScbXWq39YAICzm3cF+vHvK7+sfO4ST0VKgXxG63OWgmHl/MqCyrEEniT5grvE 5AZ+4NW9KqT7h4R0sCaGsS4HSkoVmBYOfo5sri5IH3a7v6TG+A+5GJMqcwdQx/RJb86kKWbCWdj frxzvgD4QiU1Z3ivcdgHaKmGpSuIEPQhtQZTJ8UpZRolkM1u7PVLwYZAKYfU3YaFlQ== X-Google-Smtp-Source: AGHT+IEHfmKmIygUIDhQvESzN7Wtk2XUBKOyHKoP9XCChRmGG8nn+dNRuy+maETDFRZs5Kc7YyN9oQ== X-Received: by 2002:a17:90a:ec85:b0:2f1:3355:4a8f with SMTP id 98e67ed59e1d1-2fa23f423d9mr21205213a91.4.1739209442000; Mon, 10 Feb 2025 09:44:02 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.44.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:44:01 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 13/19] bpf: net_sched: Support updating qstats Date: Mon, 10 Feb 2025 09:43:27 -0800 Message-ID: <20250210174336.2024258-14-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Allow bpf qdisc programs to update Qdisc qstats directly with btf struct access. Signed-off-by: Amery Hung --- net/sched/bpf_qdisc.c | 53 ++++++++++++++++++++++++++++++++++++------- 1 file changed, 45 insertions(+), 8 deletions(-) diff --git a/net/sched/bpf_qdisc.c b/net/sched/bpf_qdisc.c index edf01f3f1c2a..6ad3050275a4 100644 --- a/net/sched/bpf_qdisc.c +++ b/net/sched/bpf_qdisc.c @@ -36,6 +36,7 @@ bpf_qdisc_get_func_proto(enum bpf_func_id func_id, } } +BTF_ID_LIST_SINGLE(bpf_qdisc_ids, struct, Qdisc) BTF_ID_LIST_SINGLE(bpf_sk_buff_ids, struct, sk_buff) BTF_ID_LIST_SINGLE(bpf_sk_buff_ptr_ids, struct, bpf_sk_buff_ptr) @@ -60,20 +61,37 @@ static bool bpf_qdisc_is_valid_access(int off, int size, return bpf_tracing_btf_ctx_access(off, size, type, prog, info); } -static int bpf_qdisc_btf_struct_access(struct bpf_verifier_log *log, - const struct bpf_reg_state *reg, - int off, int size) +static int bpf_qdisc_qdisc_access(struct bpf_verifier_log *log, + const struct bpf_reg_state *reg, + int off, int size) { - const struct btf_type *t, *skbt; size_t end; - skbt = btf_type_by_id(reg->btf, bpf_sk_buff_ids[0]); - t = btf_type_by_id(reg->btf, reg->btf_id); - if (t != skbt) { - bpf_log(log, "only read is supported\n"); + switch (off) { + case offsetof(struct Qdisc, qstats) ... offsetofend(struct Qdisc, qstats) - 1: + end = offsetofend(struct Qdisc, qstats); + break; + default: + bpf_log(log, "no write support to Qdisc at off %d\n", off); + return -EACCES; + } + + if (off + size > end) { + bpf_log(log, + "write access at off %d with size %d beyond the member of Qdisc ended at %zu\n", + off, size, end); return -EACCES; } + return 0; +} + +static int bpf_qdisc_sk_buff_access(struct bpf_verifier_log *log, + const struct bpf_reg_state *reg, + int off, int size) +{ + size_t end; + switch (off) { case offsetof(struct sk_buff, tstamp): end = offsetofend(struct sk_buff, tstamp); @@ -115,6 +133,25 @@ static int bpf_qdisc_btf_struct_access(struct bpf_verifier_log *log, return 0; } +static int bpf_qdisc_btf_struct_access(struct bpf_verifier_log *log, + const struct bpf_reg_state *reg, + int off, int size) +{ + const struct btf_type *t, *skbt, *qdisct; + + skbt = btf_type_by_id(reg->btf, bpf_sk_buff_ids[0]); + qdisct = btf_type_by_id(reg->btf, bpf_qdisc_ids[0]); + t = btf_type_by_id(reg->btf, reg->btf_id); + + if (t == skbt) + return bpf_qdisc_sk_buff_access(log, reg, off, size); + else if (t == qdisct) + return bpf_qdisc_qdisc_access(log, reg, off, size); + + bpf_log(log, "only read is supported\n"); + return -EACCES; +} + BTF_ID_LIST(bpf_qdisc_init_prologue_ids) BTF_ID(func, bpf_qdisc_init_prologue) From patchwork Mon Feb 10 17:43:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968272 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D77EB25A333; Mon, 10 Feb 2025 17:44:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209445; cv=none; b=crHhwl3DZPQ87UTAnt9TYv5kgexd6GHkrVMEDg5n6UvrBwf+aIuLrCqYOpcnjdumzBX6mgX5utZnxrzymI/N099zptBnH+VxJ6Du+xRRGfA9Gkhs0I8lm/DulGuPEz0t8qzQOkurhKup0Gz6bOSkETtg1uI41keCn9QiKa6S8Ps= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209445; c=relaxed/simple; bh=oVv2JtaDPIKhZsdywXA46ACT2BCLo0/oHTOdRQDrkDg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Gnhzmny+ZCXnt+mHLYW0n+d0x5EOyIA2qcxJNmXZJsH/GvWBvfSHupX2NamDxzJgkx52TLbsFWLst/PHIpCWvGIXoOM4fZ7kbnRiZSleujnE7ojTxqMrgTPr0iZeO0Gi7C7xoC+dTvYVm8SzNc+tzUnFYCTn34ezBbOnMfZVjjI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=RtKHoYX+; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RtKHoYX+" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-21f7f03d856so32555715ad.1; Mon, 10 Feb 2025 09:44:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209443; x=1739814243; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6GH5rMBh11R0Ev4mXMfCpLP6RKJCsJzPSkJfSqyVzrk=; b=RtKHoYX+uVzGwfLTfJfZtfNO4obFi8225NPB++6y/S7zcWG7JSSI+KCM5McfOHPXOn PB7HWJRlCf/FB2QE2MyU6y3lhLdRHN0FU5eOnW8pJegSLJpwah2o4n/K3nisRvJogR6D wrAEKPf9fc5NXR0o7SGCHL/kse0SfV9w97hvQexrHeh6e/KH7LYmVimWGnk3RkW2hCD5 la5TH4TH//HqSQZNgbHyQ8DIYjYEPjMMw5okS8xtqaFVLw2oXwGkjx4lBPAk616aLJ7x aMuT4fChu+cFBESmxPqq/AWwFlX43gfNiOe4frdZwCtmyqAsl/SoAJNpqqSwkq2tD/b0 XSmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209443; x=1739814243; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6GH5rMBh11R0Ev4mXMfCpLP6RKJCsJzPSkJfSqyVzrk=; b=SHNUhsfZXR9Da+EvdjVv+KAQrjQ4lDyzvPc3IFAcETR9VZSoHqrE0LK8l6g6LlrF0i 5NPmxwmkuWP0aSiyifOvQHEfAXby4iZBm3mlxgUhQKXmstGlJrW2WITvWOKBimKxQK7S slV4Vk2NkMsFmPx2e5Ar/bRbOoP24p2Cs2QZ+B/n47j2Kg28tz8u3rAzjLZ/QvAcOU4E uesQuLx2ZxXLwftag8g+TNf7EBbRiOVRoFr34zfUjCV0HmnuMWBfSmQLAOeB8oQHTGBL plnE6wL/yz7Q3wi0zdAsnKXKZ7J2RiSDEkk/U2/ycnN18MYhlcFDlzXz258RtNe/QFyr RJmA== X-Gm-Message-State: AOJu0YzFqC12GCC0B4PyF+waDSQtMIEdK7jvSsnORt8YNZocVGUL8481 Lqh3EeZZ+VfOS2yCD3xqY0AuizU+sAbTItbY/QXsFJV2c6OaiJqiWom2sIgi X-Gm-Gg: ASbGnctT1BG0zwirYwLDrC2+ebxBYaIGInOrf9P6awZd04JiMkOSiYsfr2K7kU0Xbyj dw+RZiORJukEuGKmgWCB5uSvXoZaJVQlu9DoYtVCEP9MT4CIuSo1kMxN+atxjLHDEKZeGCiOhlD 5kn/SFtbNpoY3LF5jrTPMu/IbcFQGxkTuMKRGZJ38JSdlwwdSr6080gE7xIDsmVUtB0dOHu0+q0 5bwYNx7NsJ9NpmDGffkJfc+/JwyHdVCEBnUicjRB10udq2aMDeUX1PddSBM8iEX1D3Ktc3mFH6B J/JAyLWJV1T78jRKVJao/rNa/yA2nRLQOaMmQ8aAoyyp2EVdDKqPk6ml6XRJsB5BTw== X-Google-Smtp-Source: AGHT+IHmU09V34c4ifLI9dfbFiwDRvX4p09TymP+JHRWRQpwn1l7XlOcPLgFXRKVZ+qb3TldmxANrQ== X-Received: by 2002:a17:902:ecd0:b0:21f:74ec:1ff0 with SMTP id d9443c01a7336-21f74ec235fmr162670365ad.32.1739209443056; Mon, 10 Feb 2025 09:44:03 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.44.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:44:02 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 14/19] bpf: net_sched: Allow writing to more Qdisc members Date: Mon, 10 Feb 2025 09:43:28 -0800 Message-ID: <20250210174336.2024258-15-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Allow bpf qdisc to write to Qdisc->limit and Qdisc->q.qlen. Signed-off-by: Amery Hung --- net/sched/bpf_qdisc.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/net/sched/bpf_qdisc.c b/net/sched/bpf_qdisc.c index 6ad3050275a4..e4e7a5879869 100644 --- a/net/sched/bpf_qdisc.c +++ b/net/sched/bpf_qdisc.c @@ -68,6 +68,12 @@ static int bpf_qdisc_qdisc_access(struct bpf_verifier_log *log, size_t end; switch (off) { + case offsetof(struct Qdisc, limit): + end = offsetofend(struct Qdisc, limit); + break; + case offsetof(struct Qdisc, q) + offsetof(struct qdisc_skb_head, qlen): + end = offsetof(struct Qdisc, q) + offsetofend(struct qdisc_skb_head, qlen); + break; case offsetof(struct Qdisc, qstats) ... offsetofend(struct Qdisc, qstats) - 1: end = offsetofend(struct Qdisc, qstats); break; From patchwork Mon Feb 10 17:43:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968273 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F603255E24; Mon, 10 Feb 2025 17:44:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209447; cv=none; b=cUluBES+mMC4/Gec7NfzcM2kFPw5yJNrSsOfCtENtp74QHEeywJcTP4zxNdnyguWaIQhXYiZCkedR1NUNN4KzjEP/KXwf6ceAabokOQefEm7uyuhQzo1wFw7cvYxxbNCKw2E8h9ayvXIItRCih/ohQotGtpfSVUfDnALF8qKxxU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209447; c=relaxed/simple; bh=70fK2D1VKNnq07QhB2X+UwPEhOlQN3WCUcZ4aBXmKGs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MAQytrsiTruVUByI8LvXN5iORiZbGqP4uTAj3mlbiF2wn4VtcGiUXmECUkg2pHrTHs69ciAybTkEflrorRYF9/IcZgFGBceLJYyWqYzaKsyf+nprCfOn04mLZUOQwM54KxTGSbCnKy2dvS1sOhuP/TtseDaaHmgjwgtyBjdrKgw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=iy8+GV85; arc=none smtp.client-ip=209.85.216.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iy8+GV85" Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-2fa1a3c88c5so5856251a91.3; Mon, 10 Feb 2025 09:44:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209445; x=1739814245; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aFDTynPkfE2pHRJC+9xprWx2iQpT2oTsma7USZIPR7A=; b=iy8+GV85PskrQvMKjfM8fSngEIZutVhDE+WTz7T2Bv0Kf4DBGGFSfy2BxeOOmssycs cdzyYJHrCMA5q0YsUuXHokPNhb3k8WPTuOHlAH8HI0ErR2D7qGNUWP0VOyh1YoRudYni GnQypO4ctnBdYdE73Lh+Ny/XDp9kIElnWKzyBU85Y/Yqow8ChA6c3NixBEUc6Pl5A4qR 1hF5LyvD7n65AoFpfChQCTC+wQW4aZ3bs5QedAW95oMhtuq62AxxO4AUCS8E4uLsSbAb tKFtW6Q0hySdM54GdEUTXW6SFT7gjLSV7E4ro/qjwiMuFUT18i/JLgKN3vzr8HILPduQ N62A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209445; x=1739814245; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aFDTynPkfE2pHRJC+9xprWx2iQpT2oTsma7USZIPR7A=; b=T91YDJHRFgzIa8cge825BjI0ZBxokR05iQh7bHk/HwHJxmLluz3OxFher9QGWnJNPk k/kdOjwNnk5IAw8i4mqNB/tokJtyMBtTX/o9Q10LwMpQjAo5sIcbLtYCihvBju+Z5adE XRUuGmx8mohghAuhgR/lWf7Opy6kdC3GGN1bioOL2vIuX92SC8isoLl2gGAyku58yVpD 7VDWwZgk1Ni5bZ8R/fg/iarRKZdGF9UKo8oIOAFkum4V8oJ+ri+9etIiIyfZaihVIIhN MD7g4WMYHsQ6gs0CBNM9kAVv6y5aDQb4NVnfiYVwetG4eXU4q2QcTAwjccy99MUIZ1S5 oRQw== X-Gm-Message-State: AOJu0YzWArZ9Y+0cOMYdSY47NPWgrzde7Dmis12JHE9cbtl4I9LNzsg7 nRAis0CG4Zq5WjVxBwF26qrqudXW49xWHlAHBJR8mvmgwjEJ80ox237N28yR X-Gm-Gg: ASbGncsvYjVMqtG8fuUvZZCTSyr5PfPbGe9U8Jexu0YDnxl0gkkezt2G8E217TqE7xE wsrGqatYkJEvVMg/FFXViBffGXvK0JdfNQ56VN3ry0pFJBy2KHcTCOfYH1ldG9hNVcK5N8rtWoi xChIjWUKI7fwu7ctW91BjB7dFi4/ET+ESYWBlJI1u8joVXfz43XG/m8CNOrm7cfUjWr784J07I5 fN8rtmeJ5pkN7tQvmKPoHeJoCOpvGRYhl+DJAx3uOvSv3hYdUw6jGABGr3Oi/3DxaobGO+pU0yt x4u4uckR5W2ZpgPx0i6tofoOTTB8U+R9VFQjagSi2bMe6IsykThmrn85Xx2pDt2uxQ== X-Google-Smtp-Source: AGHT+IHQdIsQnjXidn38NrupTsTDhwUte5TzKjxyFG6wFnYd2ISop6Kb1umbQ+VzZ/l+yirz/I3BGQ== X-Received: by 2002:a17:90b:3144:b0:2ee:8008:b583 with SMTP id 98e67ed59e1d1-2fa9eda446cmr634475a91.16.1739209444142; Mon, 10 Feb 2025 09:44:04 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.44.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:44:03 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 15/19] bpf: net_sched: Disable attaching bpf qdisc to non root Date: Mon, 10 Feb 2025 09:43:29 -0800 Message-ID: <20250210174336.2024258-16-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Do not allow users to attach bpf qdiscs to classful qdiscs. This is to prevent accidentally breaking existings classful qdiscs if they rely on some data in the child qdisc. This restriction can potentially be lifted in the future. Note that, we still allow bpf qdisc to be attached to mq. Signed-off-by: Amery Hung Reviewed-by: Jakub Kicinski --- net/sched/bpf_qdisc.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/net/sched/bpf_qdisc.c b/net/sched/bpf_qdisc.c index e4e7a5879869..c2f33cd35674 100644 --- a/net/sched/bpf_qdisc.c +++ b/net/sched/bpf_qdisc.c @@ -170,8 +170,11 @@ static int bpf_qdisc_gen_prologue(struct bpf_insn *insn_buf, bool direct_write, return 0; *insn++ = BPF_MOV64_REG(BPF_REG_6, BPF_REG_1); + *insn++ = BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, 16); *insn++ = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0); *insn++ = BPF_CALL_KFUNC(0, bpf_qdisc_init_prologue_ids[0]); + *insn++ = BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 1); + *insn++ = BPF_EXIT_INSN(); *insn++ = BPF_MOV64_REG(BPF_REG_1, BPF_REG_6); *insn++ = prog->insnsi[0]; @@ -239,11 +242,26 @@ __bpf_kfunc void bpf_qdisc_watchdog_schedule(struct Qdisc *sch, u64 expire, u64 } /* bpf_qdisc_init_prologue - Hidden kfunc called in prologue of .init. */ -__bpf_kfunc void bpf_qdisc_init_prologue(struct Qdisc *sch) +__bpf_kfunc int bpf_qdisc_init_prologue(struct Qdisc *sch, + struct netlink_ext_ack *extack) { struct bpf_sched_data *q = qdisc_priv(sch); + struct net_device *dev = qdisc_dev(sch); + struct Qdisc *p; + + if (sch->parent != TC_H_ROOT) { + p = qdisc_lookup(dev, TC_H_MAJ(sch->parent)); + if (!p) + return -ENOENT; + + if (!(p->flags & TCQ_F_MQROOT)) { + NL_SET_ERR_MSG(extack, "BPF qdisc only supported on root or mq"); + return -EINVAL; + } + } qdisc_watchdog_init(&q->watchdog, sch); + return 0; } /* bpf_qdisc_reset_destroy_epilogue - Hidden kfunc called in epilogue of .reset From patchwork Mon Feb 10 17:43:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968275 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E87325A333; Mon, 10 Feb 2025 17:44:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209449; cv=none; b=Tt31W51KHaPxwVdTChpfGhAlgUrM7NiRmP4OMQlBCNKoi5Ml7jJdrQEGZX90StC0ZeeZ99pSwJsCUhstE/zqTluCGaw+uNfcAWx92MiPKzvFpb4MHBaXZ23nlF2CjWbWfp4OeV/B7dGWZrwy3YZ9VAvAG9CQsHXW4pWixiQIL5w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209449; c=relaxed/simple; bh=wecBTFOKU/7DDslMJtQcO1DcEyTkzBNNAGp1ShemhsA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dzMNcHJx9+/kLpJojG5Tyr0tAvKADgKe0Pzqtfau5yRXuWywU4U6t1N+W8Iuk9hGHPXB3MU79yaSfORFHdBZf9soHSWNkBDX2XPNb7IaFZdd6iIDTFwqhOEp3TtkhbPWh3/BAG3ZIkOS2y716/AEim9a8NQvNS+HJReYaG/jxQM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=FMfc0qdW; arc=none smtp.client-ip=209.85.216.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FMfc0qdW" Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-2fa21145217so7203950a91.3; Mon, 10 Feb 2025 09:44:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209447; x=1739814247; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EeAP7Bc6x9e9oxdbqzfBkmCuQiCzPgcgtTAZRyBQYU8=; b=FMfc0qdWD+p5k4WEwNzpibwFZeHxuq8nyNM+t9BiTNWGQhVXR7YNNDlERaMAjnsMxt 1YBobMq/qUIEqrTr9jBgplQIgjIwCd30bKB6jLJTmnPM+4Yml/pqdm5d/F6bkrtgSN55 DesTmJ1vV//UtirZyRT/1cm0dYPq3RJmvtfYhScDSgfKZqGelwW2JkEqwhKpgXZpqz8O YJiaOKC1zNULB+WFKyofegegLhBsF7t/uQtSGqkhqlowDTREBLQXz3G8TpCN8w7b2pyY ZKyTf/F3/ReOPxqPk0KR0TShohiVifr7PnabgJsdvvH+Xeie6h3Iz+5jMGgiBo9sSRMW ZyWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209447; x=1739814247; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EeAP7Bc6x9e9oxdbqzfBkmCuQiCzPgcgtTAZRyBQYU8=; b=RR4WAuwRkLGOsM+nIEsQHGzhe5qaVrbBU0sm1zvGulnp5uNAZcZlGyNo5VGmEqfAB7 3y8xi6drujzHJ2NWNzqD9NdPDZpdIuYG1FRSwlfw2m9ZAVLg1KoyJN8luj4Pk2/hN19H iTmMMvyFZ5n5tkw3rh/f8bdjlIDTvUZfpt0JtVD0ZtFd/mdIes93qeQfrgAXR8Ej7cXO 4NGjQvciY1ZLU0TRViOmlC4WzJh4zd3OD9L4hQ12R0pCBIX9+QRd4TZxhgDcs1kR4dsJ tbSnHxprAk0RJUKaXEQSrXp4zKrx+nHZsq0XV9FEH0pIEisqqzI2T7QOE7hqwH853878 xUww== X-Gm-Message-State: AOJu0YweVOMCI8pWegTnHufWIbdBtk9YbHHcAXlX2rqt1oblSWPR2Dcc ODKBMOWVwvPOQ0QCDH4vlrCp2wWZsBl4dUzCGHQE5ajPqn/dwpZNO68zO3Uw X-Gm-Gg: ASbGncvI3HZAQUJI4zBMbI5tivaxvpLz/QVLoEobKfdarhGdd82D9wPDk/PNRBzKjTf 4ZltLp6Jv2X4YAqQj1xlIO+cxE3hePhI0UEhX4cG7nK7sZ+nq7gStAdDj4w4pkgc4VGDmT2T7LW wL14zPabINGwRRho79TO4fWgJceAyohfO5I3W0/gYgo8A1eoD8qyKBlc9otnkFDDklBPL0Bwe5x e0EtcCllg6o/8eRVDmcYiWtgaj3K03XqMbrIEXevzPGYSFPE3yH+wfG6pZUC5HtVNArrYbL9PpQ WIEswJBQGXP8sFNNMZcK0Ozjag7gMdglB+WgrDQYNhZjOeEBUb7WDTmKobXFdkUT7A== X-Google-Smtp-Source: AGHT+IFhTVSIxmKwGH9dmYlhO4Vcd30EKVWJYTknTtHvOBBkOa6UXK/veI/brVLTkmTkXzklbNy0Ew== X-Received: by 2002:a17:90b:2d8c:b0:2fa:2124:8782 with SMTP id 98e67ed59e1d1-2fa243db995mr22985231a91.25.1739209445174; Mon, 10 Feb 2025 09:44:05 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.44.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:44:04 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 16/19] libbpf: Support creating and destroying qdisc Date: Mon, 10 Feb 2025 09:43:30 -0800 Message-ID: <20250210174336.2024258-17-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Extend struct bpf_tc_hook with handle, qdisc name and a new attach type, BPF_TC_QDISC, to allow users to add or remove any qdisc specified in addition to clsact. Signed-off-by: Amery Hung --- tools/lib/bpf/libbpf.h | 5 ++++- tools/lib/bpf/netlink.c | 20 +++++++++++++++++--- 2 files changed, 21 insertions(+), 4 deletions(-) diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index 3020ee45303a..12c81e6da219 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -1270,6 +1270,7 @@ enum bpf_tc_attach_point { BPF_TC_INGRESS = 1 << 0, BPF_TC_EGRESS = 1 << 1, BPF_TC_CUSTOM = 1 << 2, + BPF_TC_QDISC = 1 << 3, }; #define BPF_TC_PARENT(a, b) \ @@ -1284,9 +1285,11 @@ struct bpf_tc_hook { int ifindex; enum bpf_tc_attach_point attach_point; __u32 parent; + __u32 handle; + const char *qdisc; size_t :0; }; -#define bpf_tc_hook__last_field parent +#define bpf_tc_hook__last_field qdisc struct bpf_tc_opts { size_t sz; diff --git a/tools/lib/bpf/netlink.c b/tools/lib/bpf/netlink.c index 68a2def17175..c997e69d507f 100644 --- a/tools/lib/bpf/netlink.c +++ b/tools/lib/bpf/netlink.c @@ -529,9 +529,9 @@ int bpf_xdp_query_id(int ifindex, int flags, __u32 *prog_id) } -typedef int (*qdisc_config_t)(struct libbpf_nla_req *req); +typedef int (*qdisc_config_t)(struct libbpf_nla_req *req, const struct bpf_tc_hook *hook); -static int clsact_config(struct libbpf_nla_req *req) +static int clsact_config(struct libbpf_nla_req *req, const struct bpf_tc_hook *hook) { req->tc.tcm_parent = TC_H_CLSACT; req->tc.tcm_handle = TC_H_MAKE(TC_H_CLSACT, 0); @@ -539,6 +539,16 @@ static int clsact_config(struct libbpf_nla_req *req) return nlattr_add(req, TCA_KIND, "clsact", sizeof("clsact")); } +static int qdisc_config(struct libbpf_nla_req *req, const struct bpf_tc_hook *hook) +{ + const char *qdisc = OPTS_GET(hook, qdisc, NULL); + + req->tc.tcm_parent = OPTS_GET(hook, parent, TC_H_ROOT); + req->tc.tcm_handle = OPTS_GET(hook, handle, 0); + + return nlattr_add(req, TCA_KIND, qdisc, strlen(qdisc) + 1); +} + static int attach_point_to_config(struct bpf_tc_hook *hook, qdisc_config_t *config) { @@ -552,6 +562,9 @@ static int attach_point_to_config(struct bpf_tc_hook *hook, return 0; case BPF_TC_CUSTOM: return -EOPNOTSUPP; + case BPF_TC_QDISC: + *config = &qdisc_config; + return 0; default: return -EINVAL; } @@ -596,7 +609,7 @@ static int tc_qdisc_modify(struct bpf_tc_hook *hook, int cmd, int flags) req.tc.tcm_family = AF_UNSPEC; req.tc.tcm_ifindex = OPTS_GET(hook, ifindex, 0); - ret = config(&req); + ret = config(&req, hook); if (ret < 0) return ret; @@ -639,6 +652,7 @@ int bpf_tc_hook_destroy(struct bpf_tc_hook *hook) case BPF_TC_INGRESS: case BPF_TC_EGRESS: return libbpf_err(__bpf_tc_detach(hook, NULL, true)); + case BPF_TC_QDISC: case BPF_TC_INGRESS | BPF_TC_EGRESS: return libbpf_err(tc_qdisc_delete(hook)); case BPF_TC_CUSTOM: From patchwork Mon Feb 10 17:43:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968274 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 392872566C4; Mon, 10 Feb 2025 17:44:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209448; cv=none; b=aw0Q9QADdn2TIGFslsfos8FanJUcwCCYkj/w2GAkHlS5BdmNDvcSJD+xBpWeT66m2OB3eRK3kDIoweZ53wet3KizUnPQHF6KragMgKxPxIu6vSlu4C9/nKDLs6A5nUzXrNzbrNjQ2sqsZhRODmD21w+ZIaC3mWfrc94IQHT+DCE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209448; c=relaxed/simple; bh=RFTb05MOTvQ1XNo5zRefdOckyFOYcGFeYSohdaoDjRI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=c+M+R1a7DJsQL8zZJYNtFRGiaKJ0dD5ajh7MFXGQ2aexEhF/k0b8/VIC8MoWLPG8toULHfVArwcjpSMv+ievuwnQJHpokH4c0yq8VoaJJvzpo8CUXsrZBMfZ0VS4peJjnSUrPbSeYuk0T5m4zvW/FUBIYjM7GV+zFP6cUt0XKog= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=UfygcYOv; arc=none smtp.client-ip=209.85.216.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UfygcYOv" Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-2fa48404207so4257359a91.1; Mon, 10 Feb 2025 09:44:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209446; x=1739814246; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uj2Ol0TrAZxxMAWaqZASaKyaG7rHi/lbAJ67oVFmDO8=; b=UfygcYOvslAgwfMZg2UCWFEZIYpHDEJQAHmOnFWDRcQnoDwqXo0IFqN1sgi8e3WePw +n7BFWvJJ9oWuQdEP0MDCn0uBYY4ZrTFmWV1TEDB463EdyJUJBOqqHISrzjThFwq0pc6 A9xjh0pSwlt7ghGdVmimrNY2YU050J7eXWWtnI2E8Nv9IeUtps9GGB3o7T3JSoBRZOtR dnJ9lhEjs6Oz5cQzzoM0BjPHXV1iM+u4QlTn+EE9C4Q/f3QFVXzfeTnLk2w8+Lm0FlLq 2/8aFrqfmJHDzQAeML/lz7cGXeqKMivUru0fM8BBMNw8DL1TrSkioYYgxOOlsUAQh4/X x+wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209446; x=1739814246; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uj2Ol0TrAZxxMAWaqZASaKyaG7rHi/lbAJ67oVFmDO8=; b=VXeFcXUyHLAdq/qAkThGEqbIWhRr96myO8AeUDNMcsSgIzTrGY63TG3qZHe81rZmFB 5nhHirAt5x+xaAbVFTL3zTyVcTOncI2yknbjgMywQ64VMGKkAKoQMT+r6LVoA4AWbP7R ovq5GEcb/3PE5loYHOmJOCMoKCg6kXh6pa2Ftel83n5UlV1Aw0IVjyFW6+Sq3uyTIw4H /PIW7EquwHsp/kBreYppgLzTcmPsS/0dH1Sdlbg/gAgGWIUqv/rHvYkSFwvMxYZXpwCm iU5awUGgabL31KvzyDGorb1AuskSrBZaz9QVjtA2u6dGZwSEuDfCHl/T6FaqrJ68QzOc eSyQ== X-Gm-Message-State: AOJu0YzspMaWx+utqvSXAnDpRVIig081iNawJq7n5N7GD85jUPUn+n3T UFYni/Uslx54OfaE1peW0A5nrhDOUd8aHCqZ+fCh7GgvDHuOrUSpW8pbphhh X-Gm-Gg: ASbGncuDrP+itq773SRAXD3HpqqM3+h0lwySlawxkedxixSA5SGA0tPnUrzyjDNWlOb XQfzv4pCcz9qdE/osst5twri0CHSiyELA+0jnD27XI+LPtyQur6qcz3YSdqNdPx4tCrxAfMahX4 PNM0dBZpMdjJdsTKxXh83zdIQ+g+wxbrCSxNSGLlVZOHZwjwIpBwg6DTI9LVmsouYljxTonOWlx syDwSOtz9c665tLAKnw5PNeRI0ygUnoRH0N2/JHQpzsoMethn8eJEwjqIwGodSqajVx1fxCZevF xuEnP3TsIfIFTxqMOlhgfObTsBXdGYdfiq5FpBdZwBk5F4Ds5t3h7Ha8jiFM2I7cRg== X-Google-Smtp-Source: AGHT+IGdt1055JAv7BZ3s4mqMoumroYQSVN/OWuBk+S0sdxJXtOcgCSk4dINsvkr26ohQ0ya7gtvBw== X-Received: by 2002:a17:90b:2888:b0:2f5:88bb:12f with SMTP id 98e67ed59e1d1-2fa24177731mr20014870a91.21.1739209446298; Mon, 10 Feb 2025 09:44:06 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.44.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:44:05 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 17/19] selftests/bpf: Add a basic fifo qdisc test Date: Mon, 10 Feb 2025 09:43:31 -0800 Message-ID: <20250210174336.2024258-18-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung This selftest includes a bare minimum fifo qdisc, which simply enqueues sk_buffs into the back of a bpf list and dequeues from the front of the list. Signed-off-by: Amery Hung --- tools/testing/selftests/bpf/config | 1 + .../selftests/bpf/prog_tests/bpf_qdisc.c | 79 ++++++++++++ .../selftests/bpf/progs/bpf_qdisc_common.h | 27 ++++ .../selftests/bpf/progs/bpf_qdisc_fifo.c | 117 ++++++++++++++++++ 4 files changed, 224 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_qdisc_common.h create mode 100644 tools/testing/selftests/bpf/progs/bpf_qdisc_fifo.c diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config index c378d5d07e02..6b0cab55bd2d 100644 --- a/tools/testing/selftests/bpf/config +++ b/tools/testing/selftests/bpf/config @@ -71,6 +71,7 @@ CONFIG_NET_IPGRE=y CONFIG_NET_IPGRE_DEMUX=y CONFIG_NET_IPIP=y CONFIG_NET_MPLS_GSO=y +CONFIG_NET_SCH_BPF=y CONFIG_NET_SCH_FQ=y CONFIG_NET_SCH_INGRESS=y CONFIG_NET_SCHED=y diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c b/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c new file mode 100644 index 000000000000..f2efc69af348 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c @@ -0,0 +1,79 @@ +#include +#include +#include + +#include "network_helpers.h" +#include "bpf_qdisc_fifo.skel.h" + +#define LO_IFINDEX 1 + +static const unsigned int total_bytes = 10 * 1024 * 1024; + +static void do_test(char *qdisc) +{ + DECLARE_LIBBPF_OPTS(bpf_tc_hook, hook, .ifindex = LO_IFINDEX, + .attach_point = BPF_TC_QDISC, + .parent = TC_H_ROOT, + .handle = 0x8000000, + .qdisc = qdisc); + int srv_fd = -1, cli_fd = -1; + int err; + + err = bpf_tc_hook_create(&hook); + if (!ASSERT_OK(err, "attach qdisc")) + return; + + srv_fd = start_server(AF_INET6, SOCK_STREAM, NULL, 0, 0); + if (!ASSERT_OK_FD(srv_fd, "start server")) + goto done; + + cli_fd = connect_to_fd(srv_fd, 0); + if (!ASSERT_OK_FD(cli_fd, "connect to client")) + goto done; + + err = send_recv_data(srv_fd, cli_fd, total_bytes); + ASSERT_OK(err, "send_recv_data"); + +done: + if (srv_fd != -1) + close(srv_fd); + if (cli_fd != -1) + close(cli_fd); + + bpf_tc_hook_destroy(&hook); +} + +static void test_fifo(void) +{ + struct bpf_qdisc_fifo *fifo_skel; + struct bpf_link *link; + + fifo_skel = bpf_qdisc_fifo__open_and_load(); + if (!ASSERT_OK_PTR(fifo_skel, "bpf_qdisc_fifo__open_and_load")) + return; + + link = bpf_map__attach_struct_ops(fifo_skel->maps.fifo); + if (!ASSERT_OK_PTR(link, "bpf_map__attach_struct_ops")) { + bpf_qdisc_fifo__destroy(fifo_skel); + return; + } + + do_test("bpf_fifo"); + + bpf_link__destroy(link); + bpf_qdisc_fifo__destroy(fifo_skel); +} + +void test_bpf_qdisc(void) +{ + struct netns_obj *netns; + + netns = netns_new("bpf_qdisc_ns", true); + if (!ASSERT_OK_PTR(netns, "netns_new")) + return; + + if (test__start_subtest("fifo")) + test_fifo(); + + netns_free(netns); +} diff --git a/tools/testing/selftests/bpf/progs/bpf_qdisc_common.h b/tools/testing/selftests/bpf/progs/bpf_qdisc_common.h new file mode 100644 index 000000000000..62a778f94908 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/bpf_qdisc_common.h @@ -0,0 +1,27 @@ +#ifndef _BPF_QDISC_COMMON_H +#define _BPF_QDISC_COMMON_H + +#define NET_XMIT_SUCCESS 0x00 +#define NET_XMIT_DROP 0x01 /* skb dropped */ +#define NET_XMIT_CN 0x02 /* congestion notification */ + +#define TC_PRIO_CONTROL 7 +#define TC_PRIO_MAX 15 + +u32 bpf_skb_get_hash(struct sk_buff *p) __ksym; +void bpf_kfree_skb(struct sk_buff *p) __ksym; +void bpf_qdisc_skb_drop(struct sk_buff *p, struct bpf_sk_buff_ptr *to_free) __ksym; +void bpf_qdisc_watchdog_schedule(struct Qdisc *sch, u64 expire, u64 delta_ns) __ksym; +void bpf_qdisc_bstats_update(struct Qdisc *sch, const struct sk_buff *skb) __ksym; + +static struct qdisc_skb_cb *qdisc_skb_cb(const struct sk_buff *skb) +{ + return (struct qdisc_skb_cb *)skb->cb; +} + +static inline unsigned int qdisc_pkt_len(const struct sk_buff *skb) +{ + return qdisc_skb_cb(skb)->pkt_len; +} + +#endif diff --git a/tools/testing/selftests/bpf/progs/bpf_qdisc_fifo.c b/tools/testing/selftests/bpf/progs/bpf_qdisc_fifo.c new file mode 100644 index 000000000000..705e7da325da --- /dev/null +++ b/tools/testing/selftests/bpf/progs/bpf_qdisc_fifo.c @@ -0,0 +1,117 @@ +#include +#include "bpf_experimental.h" +#include "bpf_qdisc_common.h" + +char _license[] SEC("license") = "GPL"; + +struct skb_node { + struct sk_buff __kptr * skb; + struct bpf_list_node node; +}; + +#define private(name) SEC(".data." #name) __hidden __attribute__((aligned(8))) + +private(A) struct bpf_spin_lock q_fifo_lock; +private(A) struct bpf_list_head q_fifo __contains(skb_node, node); + +SEC("struct_ops/bpf_fifo_enqueue") +int BPF_PROG(bpf_fifo_enqueue, struct sk_buff *skb, struct Qdisc *sch, + struct bpf_sk_buff_ptr *to_free) +{ + struct skb_node *skbn; + u32 pkt_len; + + if (sch->q.qlen == sch->limit) + goto drop; + + skbn = bpf_obj_new(typeof(*skbn)); + if (!skbn) + goto drop; + + pkt_len = qdisc_pkt_len(skb); + + sch->q.qlen++; + skb = bpf_kptr_xchg(&skbn->skb, skb); + if (skb) + bpf_qdisc_skb_drop(skb, to_free); + + bpf_spin_lock(&q_fifo_lock); + bpf_list_push_back(&q_fifo, &skbn->node); + bpf_spin_unlock(&q_fifo_lock); + + sch->qstats.backlog += pkt_len; + return NET_XMIT_SUCCESS; +drop: + bpf_qdisc_skb_drop(skb, to_free); + return NET_XMIT_DROP; +} + +SEC("struct_ops/bpf_fifo_dequeue") +struct sk_buff *BPF_PROG(bpf_fifo_dequeue, struct Qdisc *sch) +{ + struct bpf_list_node *node; + struct sk_buff *skb = NULL; + struct skb_node *skbn; + + bpf_spin_lock(&q_fifo_lock); + node = bpf_list_pop_front(&q_fifo); + bpf_spin_unlock(&q_fifo_lock); + if (!node) + return NULL; + + skbn = container_of(node, struct skb_node, node); + skb = bpf_kptr_xchg(&skbn->skb, skb); + bpf_obj_drop(skbn); + if (!skb) + return NULL; + + sch->qstats.backlog -= qdisc_pkt_len(skb); + bpf_qdisc_bstats_update(sch, skb); + sch->q.qlen--; + + return skb; +} + +SEC("struct_ops/bpf_fifo_init") +int BPF_PROG(bpf_fifo_init, struct Qdisc *sch, struct nlattr *opt, + struct netlink_ext_ack *extack) +{ + sch->limit = 1000; + return 0; +} + +SEC("struct_ops/bpf_fifo_reset") +void BPF_PROG(bpf_fifo_reset, struct Qdisc *sch) +{ + struct bpf_list_node *node; + struct skb_node *skbn; + int i; + + bpf_for(i, 0, sch->q.qlen) { + struct sk_buff *skb = NULL; + + bpf_spin_lock(&q_fifo_lock); + node = bpf_list_pop_front(&q_fifo); + bpf_spin_unlock(&q_fifo_lock); + + if (!node) + break; + + skbn = container_of(node, struct skb_node, node); + skb = bpf_kptr_xchg(&skbn->skb, skb); + if (skb) + bpf_kfree_skb(skb); + bpf_obj_drop(skbn); + } + sch->q.qlen = 0; +} + +SEC(".struct_ops") +struct Qdisc_ops fifo = { + .enqueue = (void *)bpf_fifo_enqueue, + .dequeue = (void *)bpf_fifo_dequeue, + .init = (void *)bpf_fifo_init, + .reset = (void *)bpf_fifo_reset, + .id = "bpf_fifo", +}; + From patchwork Mon Feb 10 17:43:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968277 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4FAC259486; Mon, 10 Feb 2025 17:44:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209452; cv=none; b=TUSnjm75z9GjhFGej1d0qW3uFGx/wEPCkZFrOesriDBilWrOovzIZoqVvnEfeEXgY/H9kDjFfMFXqNJLlBIIh8LjFKLu0D0d9Pf6g2ixwpR2Q8CvOGPao737TUcsHkkX6LKYek42L2RmP1M+sv0hG78Uxb+ZeEO4Uu2aEDtV890= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209452; c=relaxed/simple; bh=l/EG2Hv2+qR3NhHzVmwDAf8x8ODiqMG1OgRk1D0eEXw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dRbaaXvXNAOaaCZvXnkpwsoOPUFG12CB8m37CGtLnqvYBOoLHCQZrwaxhcnIW5mNQgNXnTpQU58HzLvhSjJNBJHtwXFMney+KUjgI0kn3t2mFNyuZz+uhEBiVIN/pPPUlc/zrKLy/ofm5IGhrxea+7OmfmJbrOA1tYB0DOn0XWw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WO1P6PS6; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WO1P6PS6" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-21f53ad05a0so53095655ad.3; Mon, 10 Feb 2025 09:44:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209448; x=1739814248; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=f4h19YQiBcyrtSA3sf0+MDRnNgbjNX1BgikaozoMEP4=; b=WO1P6PS6w2rZOXTHAwFsOjh+mOJ0ZUOy6MdFnggqSPctrF+A1AVE+M/BrtR1xDRTdO FOl0iriLsd8erj6tunIIkl7YHmp9kjJBhxF88bfIakQTFKmxvR2+GIFtIiGxb3P11zd/ WALuqkBwD+mNNDRM+ysvDrOQ77VsajZi/OdDb30zsXivoezshafskmVPONhjAw5VOhXU 6MIWAiCfqUnoN0mWqkE+zpqTfAceixofJja/hUORDKEKjXRJ/RlMew541hqmjSDAF3/F Q+tgZKN4kJwY+wR/uq2oWDqg/sQh983b6JVEYg4g8oc8sb7wDMCHUJdmJT0N1yvnZnGQ mJHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209448; x=1739814248; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=f4h19YQiBcyrtSA3sf0+MDRnNgbjNX1BgikaozoMEP4=; b=M24tse5WwChjQCyrnMzi9RpT+SS02rVuXcEG7szMCokPBE97oOYloD5KYZ5an+Eu4F TncBovpOhxFkJLN5OqRKD+7ZS5B14BYVRJWP8YJZAbU36sw6jFEORCPysRcpxLZf0a0u OFb3IYq9LirRM6S65ZBNZvxh2gnoaU0iBXuKleTftjYjRBZE/yAYK6R3HhWetMnZI9Vi uX0MP4R7qxhODVVL0N05tIz/AbjCjIQxaEiQg5JJArU/UcCQK4jbjwwqmfEJvRN5u+pU fh1BsR2NN32ql42zZavRL6uDsd4kvpjPY9EH0wq7PjMtq9lD23f4NODI3ZAbjXzpwlbj kYMQ== X-Gm-Message-State: AOJu0YwufkK1nk6YOpVZ1GLExP1M1j5ljrXreWweUpCSPN+MOPHVYq/S jAEFp7K64gHLARuRdvh8uB3fGGOWr7wb/XCmwnYm26jAnPLXAiHBLOVJRc5y X-Gm-Gg: ASbGncux0JcAYnhZU/E/h3r+AmDeWoHTzbGlwcNJpPv44MXeOQhwLv9yWavBOj/T3x9 TeUgs9rtrGgMKMLG971+SfY+wzKc9WlCl4csI+vQo7YmZGDyXgLAW1WE6uawUvlPXllPt0A6Biv t27rvtjt6d7RHnXm33/pNjwihtk3iPY+9G8AqkFB9Yk9l29xahsBKWEcCZnHGrF+xoSRp+u+JB7 NzXM7V0LnG+xOEnJW6nd4212IwqnB0TAbUSfgePBaIc/6g/iTEs8kl4hKgA16OkayUDdN+kU+P9 whn4qyBSJ1W9nCPEHuhGtOI6UxHEfvKNQl6sgN46DPCQOQjwn/j9T3+bpv8nuUKHNQ== X-Google-Smtp-Source: AGHT+IE6da9ruRXCzTRhbyTqjY/Cu1t6svsJ+4XpP7UlU33L6aHvRTgiBvQS9zHg+BTWrG2E+at/kQ== X-Received: by 2002:a17:902:d481:b0:21f:6845:487a with SMTP id d9443c01a7336-21fb6441796mr4363905ad.23.1739209447493; Mon, 10 Feb 2025 09:44:07 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.44.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:44:07 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 18/19] selftests/bpf: Add a bpf fq qdisc to selftest Date: Mon, 10 Feb 2025 09:43:32 -0800 Message-ID: <20250210174336.2024258-19-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung This test implements a more sophisticated qdisc using bpf. The bpf fair- queueing (fq) qdisc gives each flow an equal chance to transmit data. It also respects the timestamp of skb for rate limiting. Signed-off-by: Amery Hung --- .../selftests/bpf/prog_tests/bpf_qdisc.c | 24 + .../selftests/bpf/progs/bpf_qdisc_fq.c | 718 ++++++++++++++++++ 2 files changed, 742 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/bpf_qdisc_fq.c diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c b/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c index f2efc69af348..7e8e3170e6b6 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c @@ -4,6 +4,7 @@ #include "network_helpers.h" #include "bpf_qdisc_fifo.skel.h" +#include "bpf_qdisc_fq.skel.h" #define LO_IFINDEX 1 @@ -64,6 +65,27 @@ static void test_fifo(void) bpf_qdisc_fifo__destroy(fifo_skel); } +static void test_fq(void) +{ + struct bpf_qdisc_fq *fq_skel; + struct bpf_link *link; + + fq_skel = bpf_qdisc_fq__open_and_load(); + if (!ASSERT_OK_PTR(fq_skel, "bpf_qdisc_fq__open_and_load")) + return; + + link = bpf_map__attach_struct_ops(fq_skel->maps.fq); + if (!ASSERT_OK_PTR(link, "bpf_map__attach_struct_ops")) { + bpf_qdisc_fq__destroy(fq_skel); + return; + } + + do_test("bpf_fq"); + + bpf_link__destroy(link); + bpf_qdisc_fq__destroy(fq_skel); +} + void test_bpf_qdisc(void) { struct netns_obj *netns; @@ -74,6 +96,8 @@ void test_bpf_qdisc(void) if (test__start_subtest("fifo")) test_fifo(); + if (test__start_subtest("fq")) + test_fq(); netns_free(netns); } diff --git a/tools/testing/selftests/bpf/progs/bpf_qdisc_fq.c b/tools/testing/selftests/bpf/progs/bpf_qdisc_fq.c new file mode 100644 index 000000000000..36ef53b10d98 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/bpf_qdisc_fq.c @@ -0,0 +1,718 @@ +#include +#include +#include +#include "bpf_experimental.h" +#include "bpf_qdisc_common.h" + +char _license[] SEC("license") = "GPL"; + +#define NSEC_PER_USEC 1000L +#define NSEC_PER_SEC 1000000000L + +#define NUM_QUEUE (1 << 20) + +struct fq_bpf_data { + u32 quantum; + u32 initial_quantum; + u32 flow_refill_delay; + u32 flow_plimit; + u64 horizon; + u32 orphan_mask; + u32 timer_slack; + u64 time_next_delayed_flow; + u64 unthrottle_latency_ns; + u8 horizon_drop; + u32 new_flow_cnt; + u32 old_flow_cnt; + u64 ktime_cache; +}; + +enum { + CLS_RET_PRIO = 0, + CLS_RET_NONPRIO = 1, + CLS_RET_ERR = 2, +}; + +struct skb_node { + u64 tstamp; + struct sk_buff __kptr * skb; + struct bpf_rb_node node; +}; + +struct fq_flow_node { + int credit; + u32 qlen; + u64 age; + u64 time_next_packet; + struct bpf_list_node list_node; + struct bpf_rb_node rb_node; + struct bpf_rb_root queue __contains(skb_node, node); + struct bpf_spin_lock lock; + struct bpf_refcount refcount; +}; + +struct dequeue_nonprio_ctx { + bool stop_iter; + u64 expire; + u64 now; +}; + +struct remove_flows_ctx { + bool gc_only; + u32 reset_cnt; + u32 reset_max; +}; + +struct unset_throttled_flows_ctx { + bool unset_all; + u64 now; +}; + +struct fq_stashed_flow { + struct fq_flow_node __kptr * flow; +}; + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __type(key, __u64); + __type(value, struct fq_stashed_flow); + __uint(max_entries, NUM_QUEUE); +} fq_nonprio_flows SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __type(key, __u64); + __type(value, struct fq_stashed_flow); + __uint(max_entries, 1); +} fq_prio_flows SEC(".maps"); + +#define private(name) SEC(".data." #name) __hidden __attribute__((aligned(8))) + +private(A) struct bpf_spin_lock fq_delayed_lock; +private(A) struct bpf_rb_root fq_delayed __contains(fq_flow_node, rb_node); + +private(B) struct bpf_spin_lock fq_new_flows_lock; +private(B) struct bpf_list_head fq_new_flows __contains(fq_flow_node, list_node); + +private(C) struct bpf_spin_lock fq_old_flows_lock; +private(C) struct bpf_list_head fq_old_flows __contains(fq_flow_node, list_node); + +private(D) struct fq_bpf_data q; + +/* Wrapper for bpf_kptr_xchg that expects NULL dst */ +static void bpf_kptr_xchg_back(void *map_val, void *ptr) +{ + void *ret; + + ret = bpf_kptr_xchg(map_val, ptr); + if (ret) + bpf_obj_drop(ret); +} + +static bool skbn_tstamp_less(struct bpf_rb_node *a, const struct bpf_rb_node *b) +{ + struct skb_node *skbn_a; + struct skb_node *skbn_b; + + skbn_a = container_of(a, struct skb_node, node); + skbn_b = container_of(b, struct skb_node, node); + + return skbn_a->tstamp < skbn_b->tstamp; +} + +static bool fn_time_next_packet_less(struct bpf_rb_node *a, const struct bpf_rb_node *b) +{ + struct fq_flow_node *flow_a; + struct fq_flow_node *flow_b; + + flow_a = container_of(a, struct fq_flow_node, rb_node); + flow_b = container_of(b, struct fq_flow_node, rb_node); + + return flow_a->time_next_packet < flow_b->time_next_packet; +} + +static void +fq_flows_add_head(struct bpf_list_head *head, struct bpf_spin_lock *lock, + struct fq_flow_node *flow, u32 *flow_cnt) +{ + bpf_spin_lock(lock); + bpf_list_push_front(head, &flow->list_node); + bpf_spin_unlock(lock); + *flow_cnt += 1; +} + +static void +fq_flows_add_tail(struct bpf_list_head *head, struct bpf_spin_lock *lock, + struct fq_flow_node *flow, u32 *flow_cnt) +{ + bpf_spin_lock(lock); + bpf_list_push_back(head, &flow->list_node); + bpf_spin_unlock(lock); + *flow_cnt += 1; +} + +static void +fq_flows_remove_front(struct bpf_list_head *head, struct bpf_spin_lock *lock, + struct bpf_list_node **node, u32 *flow_cnt) +{ + bpf_spin_lock(lock); + *node = bpf_list_pop_front(head); + bpf_spin_unlock(lock); + *flow_cnt -= 1; +} + +static bool +fq_flows_is_empty(struct bpf_list_head *head, struct bpf_spin_lock *lock) +{ + struct bpf_list_node *node; + + bpf_spin_lock(lock); + node = bpf_list_pop_front(head); + if (node) { + bpf_list_push_front(head, node); + bpf_spin_unlock(lock); + return false; + } + bpf_spin_unlock(lock); + + return true; +} + +/* flow->age is used to denote the state of the flow (not-detached, detached, throttled) + * as well as the timestamp when the flow is detached. + * + * 0: not-detached + * 1 - (~0ULL-1): detached + * ~0ULL: throttled + */ +static void fq_flow_set_detached(struct fq_flow_node *flow) +{ + flow->age = bpf_jiffies64(); +} + +static bool fq_flow_is_detached(struct fq_flow_node *flow) +{ + return flow->age != 0 && flow->age != ~0ULL; +} + +static bool sk_listener(struct sock *sk) +{ + return (1 << sk->__sk_common.skc_state) & (TCPF_LISTEN | TCPF_NEW_SYN_RECV); +} + +static void fq_gc(void); + +static int fq_new_flow(void *flow_map, struct fq_stashed_flow **sflow, u64 hash) +{ + struct fq_stashed_flow tmp = {}; + struct fq_flow_node *flow; + int ret; + + flow = bpf_obj_new(typeof(*flow)); + if (!flow) + return -ENOMEM; + + flow->credit = q.initial_quantum, + flow->qlen = 0, + flow->age = 1, + flow->time_next_packet = 0, + + ret = bpf_map_update_elem(flow_map, &hash, &tmp, 0); + if (ret == -ENOMEM || ret == -E2BIG) { + fq_gc(); + bpf_map_update_elem(&fq_nonprio_flows, &hash, &tmp, 0); + } + + *sflow = bpf_map_lookup_elem(flow_map, &hash); + if (!*sflow) { + bpf_obj_drop(flow); + return -ENOMEM; + } + + bpf_kptr_xchg_back(&(*sflow)->flow, flow); + return 0; +} + +static int +fq_classify(struct sk_buff *skb, struct fq_stashed_flow **sflow) +{ + struct sock *sk = skb->sk; + int ret = CLS_RET_NONPRIO; + u64 hash = 0; + + if ((skb->priority & TC_PRIO_MAX) == TC_PRIO_CONTROL) { + *sflow = bpf_map_lookup_elem(&fq_prio_flows, &hash); + ret = CLS_RET_PRIO; + } else { + if (!sk || sk_listener(sk)) { + hash = bpf_skb_get_hash(skb) & q.orphan_mask; + /* Avoid collision with an existing flow hash, which + * only uses the lower 32 bits of hash, by setting the + * upper half of hash to 1. + */ + hash |= (1ULL << 32); + } else if (sk->__sk_common.skc_state == TCP_CLOSE) { + hash = bpf_skb_get_hash(skb) & q.orphan_mask; + hash |= (1ULL << 32); + } else { + hash = sk->__sk_common.skc_hash; + } + *sflow = bpf_map_lookup_elem(&fq_nonprio_flows, &hash); + } + + if (!*sflow) + ret = fq_new_flow(&fq_nonprio_flows, sflow, hash) < 0 ? + CLS_RET_ERR : CLS_RET_NONPRIO; + + return ret; +} + +static bool fq_packet_beyond_horizon(struct sk_buff *skb) +{ + return (s64)skb->tstamp > (s64)(q.ktime_cache + q.horizon); +} + +SEC("struct_ops/bpf_fq_enqueue") +int BPF_PROG(bpf_fq_enqueue, struct sk_buff *skb, struct Qdisc *sch, + struct bpf_sk_buff_ptr *to_free) +{ + struct fq_flow_node *flow = NULL, *flow_copy; + struct fq_stashed_flow *sflow; + u64 time_to_send, jiffies; + struct skb_node *skbn; + int ret; + + if (sch->q.qlen >= sch->limit) + goto drop; + + if (!skb->tstamp) { + time_to_send = q.ktime_cache = bpf_ktime_get_ns(); + } else { + if (fq_packet_beyond_horizon(skb)) { + q.ktime_cache = bpf_ktime_get_ns(); + if (fq_packet_beyond_horizon(skb)) { + if (q.horizon_drop) + goto drop; + + skb->tstamp = q.ktime_cache + q.horizon; + } + } + time_to_send = skb->tstamp; + } + + ret = fq_classify(skb, &sflow); + if (ret == CLS_RET_ERR) + goto drop; + + flow = bpf_kptr_xchg(&sflow->flow, flow); + if (!flow) + goto drop; + + if (ret == CLS_RET_NONPRIO) { + if (flow->qlen >= q.flow_plimit) { + bpf_kptr_xchg_back(&sflow->flow, flow); + goto drop; + } + + if (fq_flow_is_detached(flow)) { + flow_copy = bpf_refcount_acquire(flow); + + jiffies = bpf_jiffies64(); + if ((s64)(jiffies - (flow_copy->age + q.flow_refill_delay)) > 0) { + if (flow_copy->credit < q.quantum) + flow_copy->credit = q.quantum; + } + flow_copy->age = 0; + fq_flows_add_tail(&fq_new_flows, &fq_new_flows_lock, flow_copy, + &q.new_flow_cnt); + } + } + + skbn = bpf_obj_new(typeof(*skbn)); + if (!skbn) { + bpf_kptr_xchg_back(&sflow->flow, flow); + goto drop; + } + + skbn->tstamp = skb->tstamp = time_to_send; + + sch->qstats.backlog += qdisc_pkt_len(skb); + + skb = bpf_kptr_xchg(&skbn->skb, skb); + if (skb) + bpf_qdisc_skb_drop(skb, to_free); + + bpf_spin_lock(&flow->lock); + bpf_rbtree_add(&flow->queue, &skbn->node, skbn_tstamp_less); + bpf_spin_unlock(&flow->lock); + + flow->qlen++; + bpf_kptr_xchg_back(&sflow->flow, flow); + + sch->q.qlen++; + return NET_XMIT_SUCCESS; + +drop: + bpf_qdisc_skb_drop(skb, to_free); + sch->qstats.drops++; + return NET_XMIT_DROP; +} + +static int fq_unset_throttled_flows(u32 index, struct unset_throttled_flows_ctx *ctx) +{ + struct bpf_rb_node *node = NULL; + struct fq_flow_node *flow; + + bpf_spin_lock(&fq_delayed_lock); + + node = bpf_rbtree_first(&fq_delayed); + if (!node) { + bpf_spin_unlock(&fq_delayed_lock); + return 1; + } + + flow = container_of(node, struct fq_flow_node, rb_node); + if (!ctx->unset_all && flow->time_next_packet > ctx->now) { + q.time_next_delayed_flow = flow->time_next_packet; + bpf_spin_unlock(&fq_delayed_lock); + return 1; + } + + node = bpf_rbtree_remove(&fq_delayed, &flow->rb_node); + + bpf_spin_unlock(&fq_delayed_lock); + + if (!node) + return 1; + + flow = container_of(node, struct fq_flow_node, rb_node); + flow->age = 0; + fq_flows_add_tail(&fq_old_flows, &fq_old_flows_lock, flow, &q.old_flow_cnt); + + return 0; +} + +static void fq_flow_set_throttled(struct fq_flow_node *flow) +{ + flow->age = ~0ULL; + + if (q.time_next_delayed_flow > flow->time_next_packet) + q.time_next_delayed_flow = flow->time_next_packet; + + bpf_spin_lock(&fq_delayed_lock); + bpf_rbtree_add(&fq_delayed, &flow->rb_node, fn_time_next_packet_less); + bpf_spin_unlock(&fq_delayed_lock); +} + +static void fq_check_throttled(u64 now) +{ + struct unset_throttled_flows_ctx ctx = { + .unset_all = false, + .now = now, + }; + unsigned long sample; + + if (q.time_next_delayed_flow > now) + return; + + sample = (unsigned long)(now - q.time_next_delayed_flow); + q.unthrottle_latency_ns -= q.unthrottle_latency_ns >> 3; + q.unthrottle_latency_ns += sample >> 3; + + q.time_next_delayed_flow = ~0ULL; + bpf_loop(NUM_QUEUE, fq_unset_throttled_flows, &ctx, 0); +} + +static struct sk_buff* +fq_dequeue_nonprio_flows(u32 index, struct dequeue_nonprio_ctx *ctx) +{ + u64 time_next_packet, time_to_send; + struct bpf_rb_node *rb_node; + struct sk_buff *skb = NULL; + struct bpf_list_head *head; + struct bpf_list_node *node; + struct bpf_spin_lock *lock; + struct fq_flow_node *flow; + struct skb_node *skbn; + bool is_empty; + u32 *cnt; + + if (q.new_flow_cnt) { + head = &fq_new_flows; + lock = &fq_new_flows_lock; + cnt = &q.new_flow_cnt; + } else if (q.old_flow_cnt) { + head = &fq_old_flows; + lock = &fq_old_flows_lock; + cnt = &q.old_flow_cnt; + } else { + if (q.time_next_delayed_flow != ~0ULL) + ctx->expire = q.time_next_delayed_flow; + goto break_loop; + } + + fq_flows_remove_front(head, lock, &node, cnt); + if (!node) + goto break_loop; + + flow = container_of(node, struct fq_flow_node, list_node); + if (flow->credit <= 0) { + flow->credit += q.quantum; + fq_flows_add_tail(&fq_old_flows, &fq_old_flows_lock, flow, &q.old_flow_cnt); + return NULL; + } + + bpf_spin_lock(&flow->lock); + rb_node = bpf_rbtree_first(&flow->queue); + if (!rb_node) { + bpf_spin_unlock(&flow->lock); + is_empty = fq_flows_is_empty(&fq_old_flows, &fq_old_flows_lock); + if (head == &fq_new_flows && !is_empty) { + fq_flows_add_tail(&fq_old_flows, &fq_old_flows_lock, flow, &q.old_flow_cnt); + } else { + fq_flow_set_detached(flow); + bpf_obj_drop(flow); + } + return NULL; + } + + skbn = container_of(rb_node, struct skb_node, node); + time_to_send = skbn->tstamp; + + time_next_packet = (time_to_send > flow->time_next_packet) ? + time_to_send : flow->time_next_packet; + if (ctx->now < time_next_packet) { + bpf_spin_unlock(&flow->lock); + flow->time_next_packet = time_next_packet; + fq_flow_set_throttled(flow); + return NULL; + } + + rb_node = bpf_rbtree_remove(&flow->queue, rb_node); + bpf_spin_unlock(&flow->lock); + + if (!rb_node) + goto add_flow_and_break; + + skbn = container_of(rb_node, struct skb_node, node); + skb = bpf_kptr_xchg(&skbn->skb, skb); + bpf_obj_drop(skbn); + + if (!skb) + goto add_flow_and_break; + + flow->credit -= qdisc_skb_cb(skb)->pkt_len; + flow->qlen--; + +add_flow_and_break: + fq_flows_add_head(head, lock, flow, cnt); + +break_loop: + ctx->stop_iter = true; + return skb; +} + +static struct sk_buff *fq_dequeue_prio(void) +{ + struct fq_flow_node *flow = NULL; + struct fq_stashed_flow *sflow; + struct bpf_rb_node *rb_node; + struct sk_buff *skb = NULL; + struct skb_node *skbn; + u64 hash = 0; + + sflow = bpf_map_lookup_elem(&fq_prio_flows, &hash); + if (!sflow) + return NULL; + + flow = bpf_kptr_xchg(&sflow->flow, flow); + if (!flow) + return NULL; + + bpf_spin_lock(&flow->lock); + rb_node = bpf_rbtree_first(&flow->queue); + if (!rb_node) { + bpf_spin_unlock(&flow->lock); + goto out; + } + + skbn = container_of(rb_node, struct skb_node, node); + rb_node = bpf_rbtree_remove(&flow->queue, &skbn->node); + bpf_spin_unlock(&flow->lock); + + if (!rb_node) + goto out; + + skbn = container_of(rb_node, struct skb_node, node); + skb = bpf_kptr_xchg(&skbn->skb, skb); + bpf_obj_drop(skbn); + +out: + bpf_kptr_xchg_back(&sflow->flow, flow); + + return skb; +} + +SEC("struct_ops/bpf_fq_dequeue") +struct sk_buff *BPF_PROG(bpf_fq_dequeue, struct Qdisc *sch) +{ + struct dequeue_nonprio_ctx cb_ctx = {}; + struct sk_buff *skb = NULL; + int i; + + if (!sch->q.qlen) + goto out; + + skb = fq_dequeue_prio(); + if (skb) + goto dequeue; + + q.ktime_cache = cb_ctx.now = bpf_ktime_get_ns(); + fq_check_throttled(q.ktime_cache); + bpf_for(i, 0, sch->limit) { + skb = fq_dequeue_nonprio_flows(i, &cb_ctx); + if (cb_ctx.stop_iter) + break; + }; + + if (skb) { +dequeue: + sch->q.qlen--; + sch->qstats.backlog -= qdisc_pkt_len(skb); + bpf_qdisc_bstats_update(sch, skb); + return skb; + } + + if (cb_ctx.expire) + bpf_qdisc_watchdog_schedule(sch, cb_ctx.expire, q.timer_slack); +out: + return NULL; +} + +static int fq_remove_flows_in_list(u32 index, void *ctx) +{ + struct bpf_list_node *node; + struct fq_flow_node *flow; + + bpf_spin_lock(&fq_new_flows_lock); + node = bpf_list_pop_front(&fq_new_flows); + bpf_spin_unlock(&fq_new_flows_lock); + if (!node) { + bpf_spin_lock(&fq_old_flows_lock); + node = bpf_list_pop_front(&fq_old_flows); + bpf_spin_unlock(&fq_old_flows_lock); + if (!node) + return 1; + } + + flow = container_of(node, struct fq_flow_node, list_node); + bpf_obj_drop(flow); + + return 0; +} + +extern unsigned CONFIG_HZ __kconfig; + +/* limit number of collected flows per round */ +#define FQ_GC_MAX 8 +#define FQ_GC_AGE (3*CONFIG_HZ) + +static bool fq_gc_candidate(struct fq_flow_node *flow) +{ + u64 jiffies = bpf_jiffies64(); + + return fq_flow_is_detached(flow) && + ((s64)(jiffies - (flow->age + FQ_GC_AGE)) > 0); +} + +static int +fq_remove_flows(struct bpf_map *flow_map, u64 *hash, + struct fq_stashed_flow *sflow, struct remove_flows_ctx *ctx) +{ + if (sflow->flow && + (!ctx->gc_only || fq_gc_candidate(sflow->flow))) { + bpf_map_delete_elem(flow_map, hash); + ctx->reset_cnt++; + } + + return ctx->reset_cnt < ctx->reset_max ? 0 : 1; +} + +static void fq_gc(void) +{ + struct remove_flows_ctx cb_ctx = { + .gc_only = true, + .reset_cnt = 0, + .reset_max = FQ_GC_MAX, + }; + + bpf_for_each_map_elem(&fq_nonprio_flows, fq_remove_flows, &cb_ctx, 0); +} + +SEC("struct_ops/bpf_fq_reset") +void BPF_PROG(bpf_fq_reset, struct Qdisc *sch) +{ + struct unset_throttled_flows_ctx utf_ctx = { + .unset_all = true, + }; + struct remove_flows_ctx rf_ctx = { + .gc_only = false, + .reset_cnt = 0, + .reset_max = NUM_QUEUE, + }; + struct fq_stashed_flow *sflow; + u64 hash = 0; + + sch->q.qlen = 0; + sch->qstats.backlog = 0; + + bpf_for_each_map_elem(&fq_nonprio_flows, fq_remove_flows, &rf_ctx, 0); + + rf_ctx.reset_cnt = 0; + bpf_for_each_map_elem(&fq_prio_flows, fq_remove_flows, &rf_ctx, 0); + fq_new_flow(&fq_prio_flows, &sflow, hash); + + bpf_loop(NUM_QUEUE, fq_remove_flows_in_list, NULL, 0); + q.new_flow_cnt = 0; + q.old_flow_cnt = 0; + + bpf_loop(NUM_QUEUE, fq_unset_throttled_flows, &utf_ctx, 0); +} + +SEC("struct_ops/bpf_fq_init") +int BPF_PROG(bpf_fq_init, struct Qdisc *sch, struct nlattr *opt, + struct netlink_ext_ack *extack) +{ + struct net_device *dev = sch->dev_queue->dev; + u32 psched_mtu = dev->mtu + dev->hard_header_len; + struct fq_stashed_flow *sflow; + u64 hash = 0; + + if (fq_new_flow(&fq_prio_flows, &sflow, hash) < 0) + return -ENOMEM; + + sch->limit = 10000; + q.initial_quantum = 10 * psched_mtu; + q.quantum = 2 * psched_mtu; + q.flow_refill_delay = 40; + q.flow_plimit = 100; + q.horizon = 10ULL * NSEC_PER_SEC; + q.horizon_drop = 1; + q.orphan_mask = 1024 - 1; + q.timer_slack = 10 * NSEC_PER_USEC; + q.time_next_delayed_flow = ~0ULL; + q.unthrottle_latency_ns = 0ULL; + q.new_flow_cnt = 0; + q.old_flow_cnt = 0; + + return 0; +} + +SEC(".struct_ops") +struct Qdisc_ops fq = { + .enqueue = (void *)bpf_fq_enqueue, + .dequeue = (void *)bpf_fq_dequeue, + .reset = (void *)bpf_fq_reset, + .init = (void *)bpf_fq_init, + .id = "bpf_fq", +}; From patchwork Mon Feb 10 17:43:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13968276 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33EB5257ACE; Mon, 10 Feb 2025 17:44:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209450; cv=none; b=QEqwLAesS+m8xBt12JP/2WNYH2BSvgtf9XIfqgPcyJKi5b+DQWpGdLM0kBphctddye76xRzxJUlOmbBlVTB9gCkQ1eDkNyNWZWHekp1xdptNPr0uGJf2efAd2e96ArrhMCLbNlZy+GusX6+Pi5v6l+zwhDdEXgOhyluuL1Q10x8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739209450; c=relaxed/simple; bh=sVVkbk0Gn1THhmgFQQYZXWJ516ceqWV7SjuSKbWsCVg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eJgQAForLUlK1dmB2jvTmgUx5iJjmg3nTSaq03N+J4+vK3eMsimO5Cn2sUhZo9T9xOqgup6wT0jflmGKGvFeGOvQfRcPC9SHelD3WwlRs6FZXjkBYJ4Rl82AJypKfN7dbRg9qL825povtDnWT5JFwPUuDkRU9p/AeG6/njzcHKM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OrgBZ2H9; arc=none smtp.client-ip=209.85.216.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OrgBZ2H9" Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-2fa40c0bab2so4582396a91.0; Mon, 10 Feb 2025 09:44:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739209448; x=1739814248; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tCot4AmKUdSjbCpz+84y1dF8CcLEygWZ6VcGHROBKi0=; b=OrgBZ2H9xgqfRzwspedh5cy7sxDwbimO7B/BppDOsiwahGWgIF92V6TDXQftrymsgZ GYuflqUMwIjBBKfLSrsStv/0HCCnLY11o3owE/1D/svRcMQb0jDgu6fZfbgIt9Wim3od DcwgBKENGU4Z33EB+eQ8zcy//tStSZc6Jw5W7FEyJ8teXq7FAG1/Ybcl7hgs5/hihG9g 2RzKMadEI/dN3sLbRTGvWsPrNlFxs4hO+Cljrd7FDQijEKmM1H60XBMmSkrrzQVDJxT8 WQnlEl0rZEzKSeNfDfpAZRZdO5rMua8qaTcS9n4DXNU6wEw3cWYdvMaecd8tS2u3nOjH 10Iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739209448; x=1739814248; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tCot4AmKUdSjbCpz+84y1dF8CcLEygWZ6VcGHROBKi0=; b=CSK6bi4PbXgTgg6eoDwLp76ZgXf5VjwTlcEVkH+syuD+QyBX3Mo4dkGggOOMweIhDi qvebENqx5BsON06bw7guk5NUnuT0lZsr//3Z/jwVGW3z6C7qZd/tr71zdouM97QQ0FrX U2pKgexMN6p+QodAFLCuK9CtR9TuXsIUsRtHBkEtcJ0m+h32wC50FLuZcz1A4k9eXrJu miSPEOo8dZxUjFmD1VIktspvEMhXH0tMZZRCuzK1XVrvI6ZjD5OZ6874NvJBjGiC245F PbAddW8k+gROy7ajO3vsnuB1Oo5P1Et+dRVdXne7qTkOFRl712WaqsrSznvy594maexn yfxQ== X-Gm-Message-State: AOJu0YzUkutGR6w+UHYbiW1AD8FWgVuVRdVkKw6fUc79wWvssR0SYFX6 KF2X18L5JWEPdAcP9Y7Wjcb9fxU2siqC9FxGIpc+7Evf6BhwJh79KvmD6Kul X-Gm-Gg: ASbGncte6f8VXJ7UFj+raCZUW2VXapUvSkKQaY3UKEImZJfOfK5pWB93Mugqxs9fuEK VpmsFXOeDgmocpGFnL4f6/JBpygG8+CHy6r5xr6CdQ3opbqr7J9xIn3ClHDHioRLCl38bCmuTUk 68Vt6HSH027uwcmOhXHddVu2Gbl9H5XKWBSedXXeEkSz+XbNoZZpZnuhBUBUrxYC7SuiHM6uhjr AtOCEBdfhCZElohGCfEI9AH+N/BLovLWga5Nmba6sMBTCwCKgyd6vBDvNM+nWhUfIvy2D4FGd6J Lvngi178pHa4qc5L2xUNvlF+/Gw90vzwmLzis1/e0Rr3Udt+ZnMeTxVqXbY7vXqeTA== X-Google-Smtp-Source: AGHT+IGOQqcef8+uXJOslKW4MqjO0+yfdJIEKy/ckO4RYPSE7YgsLhAZhQKf734vkt/D81RganGRcg== X-Received: by 2002:a17:90b:350b:b0:2fa:228d:5af3 with SMTP id 98e67ed59e1d1-2fa24271b0bmr24427087a91.15.1739209448573; Mon, 10 Feb 2025 09:44:08 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2fa3fb55dcasm5554961a91.4.2025.02.10.09.44.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Feb 2025 09:44:08 -0800 (PST) From: Amery Hung To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, kuba@kernel.org, edumazet@google.com, xiyou.wangcong@gmail.com, cong.wang@bytedance.com, jhs@mojatatu.com, sinquersw@gmail.com, toke@redhat.com, jiri@resnulli.us, stfomichev@gmail.com, ekarani.silvestre@ccc.ufcg.edu.br, yangpeihao@sjtu.edu.cn, yepeilin.cs@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v4 19/19] selftests/bpf: Test attaching bpf qdisc to mq and non root Date: Mon, 10 Feb 2025 09:43:33 -0800 Message-ID: <20250210174336.2024258-20-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250210174336.2024258-1-ameryhung@gmail.com> References: <20250210174336.2024258-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Until we are certain that existing classful qdiscs work with bpf qdisc, make sure we don't allow attaching a bpf qdisc to non root. Meanwhile, attaching to mq is allowed. Signed-off-by: Amery Hung --- tools/testing/selftests/bpf/config | 1 + .../selftests/bpf/prog_tests/bpf_qdisc.c | 75 +++++++++++++++++++ 2 files changed, 76 insertions(+) diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config index 6b0cab55bd2d..3201a962b3dc 100644 --- a/tools/testing/selftests/bpf/config +++ b/tools/testing/selftests/bpf/config @@ -74,6 +74,7 @@ CONFIG_NET_MPLS_GSO=y CONFIG_NET_SCH_BPF=y CONFIG_NET_SCH_FQ=y CONFIG_NET_SCH_INGRESS=y +CONFIG_NET_SCH_HTB=y CONFIG_NET_SCHED=y CONFIG_NETDEVSIM=y CONFIG_NETFILTER=y diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c b/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c index 7e8e3170e6b6..5d4fa5ad40e1 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c @@ -86,6 +86,77 @@ static void test_fq(void) bpf_qdisc_fq__destroy(fq_skel); } +static void test_qdisc_attach_to_mq(void) +{ + DECLARE_LIBBPF_OPTS(bpf_tc_hook, hook, + .attach_point = BPF_TC_QDISC, + .parent = TC_H_MAKE(1 << 16, 1), + .handle = 0x11 << 16, + .qdisc = "bpf_fifo"); + struct bpf_qdisc_fifo *fifo_skel; + struct bpf_link *link; + int err; + + fifo_skel = bpf_qdisc_fifo__open_and_load(); + if (!ASSERT_OK_PTR(fifo_skel, "bpf_qdisc_fifo__open_and_load")) + return; + + link = bpf_map__attach_struct_ops(fifo_skel->maps.fifo); + if (!ASSERT_OK_PTR(link, "bpf_map__attach_struct_ops")) { + bpf_qdisc_fifo__destroy(fifo_skel); + return; + } + + SYS(out, "ip link add veth0 type veth peer veth1"); + hook.ifindex = if_nametoindex("veth0"); + SYS(out, "tc qdisc add dev veth0 root handle 1: mq"); + + err = bpf_tc_hook_create(&hook); + ASSERT_OK(err, "attach qdisc"); + + bpf_tc_hook_destroy(&hook); + + SYS(out, "tc qdisc delete dev veth0 root mq"); +out: + bpf_link__destroy(link); + bpf_qdisc_fifo__destroy(fifo_skel); +} + +static void test_qdisc_attach_to_non_root(void) +{ + DECLARE_LIBBPF_OPTS(bpf_tc_hook, hook, .ifindex = LO_IFINDEX, + .attach_point = BPF_TC_QDISC, + .parent = TC_H_MAKE(1 << 16, 1), + .handle = 0x11 << 16, + .qdisc = "bpf_fifo"); + struct bpf_qdisc_fifo *fifo_skel; + struct bpf_link *link; + int err; + + fifo_skel = bpf_qdisc_fifo__open_and_load(); + if (!ASSERT_OK_PTR(fifo_skel, "bpf_qdisc_fifo__open_and_load")) + return; + + link = bpf_map__attach_struct_ops(fifo_skel->maps.fifo); + if (!ASSERT_OK_PTR(link, "bpf_map__attach_struct_ops")) { + bpf_qdisc_fifo__destroy(fifo_skel); + return; + } + + SYS(out, "tc qdisc add dev lo root handle 1: htb"); + SYS(out_del_htb, "tc class add dev lo parent 1: classid 1:1 htb rate 75Kbit"); + + err = bpf_tc_hook_create(&hook); + if (!ASSERT_ERR(err, "attach qdisc")) + bpf_tc_hook_destroy(&hook); + +out_del_htb: + SYS(out, "tc qdisc delete dev lo root htb"); +out: + bpf_link__destroy(link); + bpf_qdisc_fifo__destroy(fifo_skel); +} + void test_bpf_qdisc(void) { struct netns_obj *netns; @@ -98,6 +169,10 @@ void test_bpf_qdisc(void) test_fifo(); if (test__start_subtest("fq")) test_fq(); + if (test__start_subtest("attach to mq")) + test_qdisc_attach_to_mq(); + if (test__start_subtest("attach to non root")) + test_qdisc_attach_to_non_root(); netns_free(netns); }