From patchwork Tue Dec 13 02:35:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stanislav Fomichev X-Patchwork-Id: 13071664 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5A6CC4332F for ; Tue, 13 Dec 2022 02:36:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234265AbiLMCgy (ORCPT ); Mon, 12 Dec 2022 21:36:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46120 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234145AbiLMCgd (ORCPT ); Mon, 12 Dec 2022 21:36:33 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE5C21DDF3 for ; Mon, 12 Dec 2022 18:36:17 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id r7-20020a25c107000000b006ff55ac0ee7so15099064ybf.15 for ; Mon, 12 Dec 2022 18:36:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=hHrbQKzCmMkP479Z1RRO0/pYKwPfq1SR7Hgtp7/VwOs=; b=iSFgiSo/fvsyyLVOyUFMSwI6mkJf5wRVolbeG55XCn1EGMIMSfLaCO65nyrY6mccV5 k+8NeKzWmdvwr80mBhm9HBcFnHLAKSHgJd7KcRWjPiT1F6e8JTUryriDHwgEURb7BULO tLh8rG+znedhLY5sBaqQRmcLFL5/Wieh4Q8szEWd3lohTgZtnj9ave2k0own4WA73QE9 lRn7joKrJ88Z2rPHpeb6vDMhYVmrGoWXKMT/003qNMyWT9W5TP2A1R/GL/FhjNDjs2Ja 56ARUHpdm3hA6J6pVM30A8hG1PrV6467Dms6xVXj1Cmbf+F4z6ML1iOPZV2YL+gDBVzj aSYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=hHrbQKzCmMkP479Z1RRO0/pYKwPfq1SR7Hgtp7/VwOs=; b=NhBva/Zdc9il/WelmI0ri7pxIIJS3QxUww8Z4n5hMf00T7B4yRmLwIZVaCeZbWekHi yhJAklhGluzhLZ1aiCbYxpxUPx+9wIV9NNO/25M7NiyFISwXt4QfxPR5IuF7c/arL86f FiNul1n51HlQFsoTFwemXjyUMR6Fd+CvOzHsQTxnShPYA2DU32/e12U7yM5UWB6QX/5f xg/0hAttnkNqElhQopVMRnW+ZZn0lZCHxf5EyyrOkrv992ZVDn28afSVmFYA1B3w+I42 EmoY6d7Rg+LF2VYi9uD7see6AXO4JA2rzAGUC721V86A+tDOZhdWMgzP1td4kBWSLQBu rEXA== X-Gm-Message-State: ANoB5pkUEmFB/2ISVXtmxgnFxC3vNVgxgJ7M/DkDC4l4SkQbbkUEFXTV jkq2aVX+mXLV6rpo7pX+l8RWxwIRkEhGJIKeYbNZTd5BzgvY7fya+ID+RbLBdLR1bg6+32z8hKl DsdPs0ihrF68/O8QuZGgb77TTaI9/Y+vMXjTnP6NkptD0qFbrBQ== X-Google-Smtp-Source: AA0mqf6E5S4pD2N0gkQvq2JrCPEjvGxNLzC1eAspPRyHTxmJwXIRG6Z4Wm3by1o5O/YVA3JuPAqfBgE= X-Received: from sdf.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5935]) (user=sdf job=sendgmr) by 2002:a25:b741:0:b0:703:6335:5f5d with SMTP id e1-20020a25b741000000b0070363355f5dmr12419814ybm.592.1670898977090; Mon, 12 Dec 2022 18:36:17 -0800 (PST) Date: Mon, 12 Dec 2022 18:35:56 -0800 In-Reply-To: <20221213023605.737383-1-sdf@google.com> Mime-Version: 1.0 References: <20221213023605.737383-1-sdf@google.com> X-Mailer: git-send-email 2.39.0.rc1.256.g54fd8350bd-goog Message-ID: <20221213023605.737383-7-sdf@google.com> Subject: [PATCH bpf-next v4 06/15] bpf: Support consuming XDP HW metadata from fext programs From: Stanislav Fomichev To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, " =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= " Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Toke Høiland-Jørgensen Instead of rejecting the attaching of PROG_TYPE_EXT programs to XDP programs that consume HW metadata, implement support for propagating the offload information. The extension program doesn't need to set a flag or ifindex, it these will just be propagated from the target by the verifier. We need to create a separate offload object for the extension program, though, since it can be reattached to a different program later (which means we can't just inhering the offload information from the target). An additional check is added on attach that the new target is compatible with the offload information in the extension prog. Signed-off-by: Toke Høiland-Jørgensen Signed-off-by: Stanislav Fomichev --- include/linux/bpf.h | 7 ++++++ kernel/bpf/offload.c | 52 ++++++++++++++++++++++++++++--------------- kernel/bpf/syscall.c | 8 +++++++ kernel/bpf/verifier.c | 19 +++++++++++----- 4 files changed, 62 insertions(+), 24 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index de6279725f41..ed288d18bf8d 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -2482,6 +2482,7 @@ void *bpf_dev_bound_resolve_kfunc(struct bpf_prog *prog, u32 func_id); void unpriv_ebpf_notify(int new_state); #if defined(CONFIG_NET) && defined(CONFIG_BPF_SYSCALL) +int __bpf_prog_dev_bound_init(struct bpf_prog *prog, struct net_device *netdev); int bpf_prog_dev_bound_init(struct bpf_prog *prog, union bpf_attr *attr); void bpf_dev_bound_netdev_unregister(struct net_device *dev); @@ -2516,6 +2517,12 @@ void sock_map_unhash(struct sock *sk); void sock_map_destroy(struct sock *sk); void sock_map_close(struct sock *sk, long timeout); #else +static inline int __bpf_prog_dev_bound_init(struct bpf_prog *prog, + struct net_device *netdev) +{ + return -EOPNOTSUPP; +} + static inline int bpf_prog_dev_bound_init(struct bpf_prog *prog, union bpf_attr *attr) { diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c index 3b6c9023f24d..0646dea1b0e1 100644 --- a/kernel/bpf/offload.c +++ b/kernel/bpf/offload.c @@ -188,17 +188,13 @@ static void __bpf_offload_dev_netdev_unregister(struct bpf_offload_dev *offdev, kfree(ondev); } -int bpf_prog_dev_bound_init(struct bpf_prog *prog, union bpf_attr *attr) +int __bpf_prog_dev_bound_init(struct bpf_prog *prog, struct net_device *netdev) { struct bpf_offload_netdev *ondev; struct bpf_prog_offload *offload; int err; - if (attr->prog_type != BPF_PROG_TYPE_SCHED_CLS && - attr->prog_type != BPF_PROG_TYPE_XDP) - return -EINVAL; - - if (attr->prog_flags & ~BPF_F_XDP_DEV_BOUND_ONLY) + if (!netdev) return -EINVAL; offload = kzalloc(sizeof(*offload), GFP_USER); @@ -206,14 +202,7 @@ int bpf_prog_dev_bound_init(struct bpf_prog *prog, union bpf_attr *attr) return -ENOMEM; offload->prog = prog; - - offload->netdev = dev_get_by_index(current->nsproxy->net_ns, - attr->prog_ifindex); - err = bpf_dev_offload_check(offload->netdev); - if (err) - goto err_maybe_put; - - prog->aux->offload_requested = !(attr->prog_flags & BPF_F_XDP_DEV_BOUND_ONLY); + offload->netdev = netdev; down_write(&bpf_devs_lock); ondev = bpf_offload_find_netdev(offload->netdev); @@ -236,19 +225,46 @@ int bpf_prog_dev_bound_init(struct bpf_prog *prog, union bpf_attr *attr) offload->offdev = ondev->offdev; prog->aux->offload = offload; list_add_tail(&offload->offloads, &ondev->progs); - dev_put(offload->netdev); up_write(&bpf_devs_lock); return 0; err_unlock: up_write(&bpf_devs_lock); -err_maybe_put: - if (offload->netdev) - dev_put(offload->netdev); kfree(offload); return err; } +int bpf_prog_dev_bound_init(struct bpf_prog *prog, union bpf_attr *attr) +{ + struct net_device *netdev; + int err; + + if (attr->prog_type != BPF_PROG_TYPE_SCHED_CLS && + attr->prog_type != BPF_PROG_TYPE_XDP) + return -EINVAL; + + if (attr->prog_flags & ~BPF_F_XDP_DEV_BOUND_ONLY) + return -EINVAL; + + netdev = dev_get_by_index(current->nsproxy->net_ns, attr->prog_ifindex); + if (!netdev) + return -EINVAL; + + err = bpf_dev_offload_check(netdev); + if (err) + goto out; + + prog->aux->offload_requested = !(attr->prog_flags & BPF_F_XDP_DEV_BOUND_ONLY); + + err = __bpf_prog_dev_bound_init(prog, netdev); + if (err) + goto out; + +out: + dev_put(netdev); + return err; +} + int bpf_prog_offload_verifier_prep(struct bpf_prog *prog) { struct bpf_prog_offload *offload; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 11c558be4992..8686475f0dbe 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -3021,6 +3021,14 @@ static int bpf_tracing_prog_attach(struct bpf_prog *prog, goto out_put_prog; } + if (bpf_prog_is_dev_bound(prog->aux) && + (bpf_prog_is_offloaded(tgt_prog->aux) || + !bpf_prog_is_dev_bound(tgt_prog->aux) || + !bpf_offload_dev_match(prog, tgt_prog->aux->offload->netdev))) { + err = -EINVAL; + goto out_put_prog; + } + key = bpf_trampoline_compute_key(tgt_prog, NULL, btf_id); } diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index e61fe0472b9b..5c6d6d61e57a 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -16524,11 +16524,6 @@ int bpf_check_attach_target(struct bpf_verifier_log *log, if (tgt_prog) { struct bpf_prog_aux *aux = tgt_prog->aux; - if (bpf_prog_is_dev_bound(tgt_prog->aux)) { - bpf_log(log, "Replacing device-bound programs not supported\n"); - return -EINVAL; - } - for (i = 0; i < aux->func_info_cnt; i++) if (aux->func_info[i].type_id == btf_id) { subprog = i; @@ -16789,10 +16784,22 @@ static int check_attach_btf_id(struct bpf_verifier_env *env) if (tgt_prog && prog->type == BPF_PROG_TYPE_EXT) { /* to make freplace equivalent to their targets, they need to * inherit env->ops and expected_attach_type for the rest of the - * verification + * verification; we also need to propagate the prog offload data + * for resolving kfuncs. */ env->ops = bpf_verifier_ops[tgt_prog->type]; prog->expected_attach_type = tgt_prog->expected_attach_type; + + if (bpf_prog_is_dev_bound(tgt_prog->aux)) { + if (bpf_prog_is_offloaded(tgt_prog->aux)) + return -EINVAL; + + prog->aux->dev_bound = true; + ret = __bpf_prog_dev_bound_init(prog, + tgt_prog->aux->offload->netdev); + if (ret) + return ret; + } } /* store info about the attachment target that will be used later */