From patchwork Tue Nov 16 05:42:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12621421 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A3D0C433FE for ; Tue, 16 Nov 2021 05:59:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1344E61501 for ; Tue, 16 Nov 2021 05:59:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237997AbhKPGBx (ORCPT ); Tue, 16 Nov 2021 01:01:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35624 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240169AbhKPGBm (ORCPT ); Tue, 16 Nov 2021 01:01:42 -0500 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 60B94C061230; Mon, 15 Nov 2021 21:42:43 -0800 (PST) Received: by mail-pj1-x1043.google.com with SMTP id h24so14827845pjq.2; Mon, 15 Nov 2021 21:42:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=f7/48XSbc+QYyvhIocYNNlFlwQkZZg5dYk++cl9LFjI=; b=kxTKhY3S7wY5ISxCXPhcu0fx14Ls+CFmG7WuijhxsRkI09c/aYHd2L+eSeQSg80ds6 X76pTO+ChPlH3uBCLlmEkOfpQGWgjOklrPsmiAJUkLDt97XMr+VFJrer+fRNalsdn30C BnY4b5+XfXh1e/ZjBG8LwmRyMW5n8TRqzh49D4ELFGyFJCKnwIFBg0f50D/X8gSyvCo7 bkr1VPK85dtNq2bAta1uilm9l6oFJou1+J/X3/MPtlIa7GAeI+TZyq9TqQzndNEiL6ec lE1msupB0s0ytp3LrSKvA7DSCU/xHIAJKw/I8zx/VLKVqoo4d0YWNaamy2CeYtkHhXOV Lz5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=f7/48XSbc+QYyvhIocYNNlFlwQkZZg5dYk++cl9LFjI=; b=gbwY2rQcZl+AjrZAlgWHTRVvrfO/95+WBYleLqGLpja61oT1sZ6zaoct3w/Wbw3EZx kOWt2BlSkgxuW51o1qiqi+ijze0jrfz7YGdUqsL0bXNnB7UaF6u/4ZZFmFpHbZaJWvaQ FIS1cSWA7R4BIR7kUDUGZu0a7KtM2oh7o1dW282HzKj6UccQ5bbzrnzHe7R82rN0PADq og8E5hZrD9DOQzOPcL6NeSG7yA+e/vRweMuz8BbW3kbeufYDJ3X7b9kPby71maAEd25T 2gZdeK8v7y4E6/tXAwsa57Nd/OxxcNDtwesAKbwehbInxtE1C7cmVUuuslHd9jh2UX14 Si8g== X-Gm-Message-State: AOAM530wyt4iF/6T+DiI+suayEXcnNT7TEwAZjz2sTkpU+OI2wbD25Em 99yMPkoNlZbZv3UHXkVGZ+Cvka7SA04= X-Google-Smtp-Source: ABdhPJyyejB2Wot1eo6b34zzmSb+bIPf0Lc5QZDOj5KqnfjJJA64rO4YJDbwhs69c5FZpRIfQEUFfg== X-Received: by 2002:a17:902:a510:b0:143:7fd1:b18a with SMTP id s16-20020a170902a51000b001437fd1b18amr42432776plq.2.1637041362748; Mon, 15 Nov 2021 21:42:42 -0800 (PST) Received: from localhost ([2405:201:6014:d064:3d4e:6265:800c:dc84]) by smtp.gmail.com with ESMTPSA id k20sm18029500pfc.83.2021.11.15.21.42.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 21:42:42 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Jens Axboe , Pavel Begunkov , io-uring@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Pavel Emelyanov , Alexander Mihalicyn , Andrei Vagin , criu@openvz.org, linux-fsdevel@vger.kernel.org Subject: [PATCH bpf-next v1 1/8] io_uring: Implement eBPF iterator for registered buffers Date: Tue, 16 Nov 2021 11:12:30 +0530 Message-Id: <20211116054237.100814-2-memxor@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211116054237.100814-1-memxor@gmail.com> References: <20211116054237.100814-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=8956; h=from:subject; bh=7cwbGmPSBXXPemkSOeSEyifdpu0D8KtxLknvhUQNWO0=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhk0S6sKksl/3ag5OKTN8ybRlXgYgNJ7rKWSW7Jcuy PE0a0fCJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYZNEugAKCRBM4MiGSL8RyoCRD/ 938mDGdY4hNEHcWDHx34dFo4g6s/HgP29nssd+R7u9IAorl8S3kPwFQeC0m6ZKt/nucLCRKoFco0Of ewa56i0uI+mVndzHKbh5/uVaw2qmOc0JeI6af2nZIc0t+6iPqnSTuxaQrynVTR3weH9gLEcNCPfyy/ vFjytzFWTdRhRwhA7UhGD2LKWpJCgx1vQmqqmeCZc8c/n28x42DxstkWhfqm5zGtPKoBWY5YTwEbkP PKpogHmXhhi5ZtfM8gklHOWC2sK3pQEXBIsj5YW0iXNlmWA/tt8MFkHnpmh1ozo04fLsQoiOMHOz1C 4ewIdOa2JBKN3iLeQPtvAX7FZumFmpVf6LM3rYr2002ad9VjRJbQ47Se0l2DESIyzfb7w1dojG8dNS HBs2UPUIqTbGx5NdJSWP2/AxV+fzcPNgeEYa9iKLgexVHrMh4X8tunWgRWMVNe1wNcRzOl3QYZmAeY edRUeUicY0vjiU8BOu6QTDuEeejoOY4olgNRWXII5ceqGSgPE6cF8ZaL5+eU1iCU6D3o/NTyKfMOlL Ug/8510bbCc/CDYzdvdAxQfKHz2lsv2q2DsEq5v3tGG9YVVorLZJAHuPZpj4VsdLblwQKHiU3Sr5hM alhmNM1HRGqbZ69/IkkrfZMg3RGasaTLfSrVgJA8jo5O0yszwEXDCGLFeJiQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This change adds eBPF iterator for buffers registered in io_uring ctx. It gives access to the ctx, the index of the registered buffer, and a pointer to the io_uring_ubuf itself. This allows the iterator to save info related to buffers added to an io_uring instance, that isn't easy to export using the fdinfo interface (like exact struct page composing the registered buffer). The primary usecase this is enabling is checkpoint/restore support. Note that we need to use mutex_trylock when the file is read from, in seq_start functions, as the order of lock taken is opposite of what it would be when io_uring operation reads the same file. We take seq_file->lock, then ctx->uring_lock, while io_uring would first take ctx->uring_lock and then seq_file->lock for the same ctx. This can lead to a deadlock scenario described below: CPU 0 CPU 1 vfs_read mutex_lock(&seq_file->lock) io_read mutex_lock(&ctx->uring_lock) mutex_lock(&ctx->uring_lock) # switched to mutex_trylock mutex_lock(&seq_file->lock) The trylock also protects the case where io_uring tries to read from iterator attached to itself (same ctx), where the order of locks would be: io_uring_enter mutex_lock(&ctx->uring_lock) <-----------. io_read \ seq_read \ mutex_lock(&seq_file->lock) / mutex_lock(&ctx->uring_lock) # deadlock-` In both these cases (recursive read and contended uring_lock), -EDEADLK is returned to userspace. In the future, this iterator will be extended to directly support iteration of bvec Flexible Array Member, so that when there is no corresponding VMA that maps to the registered buffer (e.g. if VMA is destroyed after pinning pages), we are able to reconstruct the registration on restore by dumping the page contents and then replaying them into a temporary mapping used for registration later. All this is out of scope for the current series however, but builds upon this iterator. Cc: Jens Axboe Cc: Pavel Begunkov Cc: io-uring@vger.kernel.org Signed-off-by: Kumar Kartikeya Dwivedi --- fs/io_uring.c | 179 +++++++++++++++++++++++++++++++++ include/linux/bpf.h | 2 + include/uapi/linux/bpf.h | 3 + tools/include/uapi/linux/bpf.h | 3 + 4 files changed, 187 insertions(+) diff --git a/fs/io_uring.c b/fs/io_uring.c index b07196b4511c..46a110989155 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -81,6 +81,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -11125,3 +11126,181 @@ static int __init io_uring_init(void) return 0; }; __initcall(io_uring_init); + +#ifdef CONFIG_BPF_SYSCALL + +BTF_ID_LIST(btf_io_uring_ids) +BTF_ID(struct, io_ring_ctx) +BTF_ID(struct, io_mapped_ubuf) + +struct bpf_io_uring_seq_info { + struct io_ring_ctx *ctx; + unsigned long index; +}; + +static int bpf_io_uring_init_seq(void *priv_data, struct bpf_iter_aux_info *aux) +{ + struct bpf_io_uring_seq_info *info = priv_data; + struct io_ring_ctx *ctx = aux->ctx; + + info->ctx = ctx; + return 0; +} + +static int bpf_io_uring_iter_attach(struct bpf_prog *prog, + union bpf_iter_link_info *linfo, + struct bpf_iter_aux_info *aux) +{ + struct io_ring_ctx *ctx; + struct fd f; + int ret; + + f = fdget(linfo->io_uring.io_uring_fd); + if (unlikely(!f.file)) + return -EBADF; + + ret = -EOPNOTSUPP; + if (unlikely(f.file->f_op != &io_uring_fops)) + goto out_fput; + + ret = -ENXIO; + ctx = f.file->private_data; + if (unlikely(!percpu_ref_tryget(&ctx->refs))) + goto out_fput; + + ret = 0; + aux->ctx = ctx; + +out_fput: + fdput(f); + return ret; +} + +static void bpf_io_uring_iter_detach(struct bpf_iter_aux_info *aux) +{ + percpu_ref_put(&aux->ctx->refs); +} + +/* io_uring iterator for registered buffers */ + +struct bpf_iter__io_uring_buf { + __bpf_md_ptr(struct bpf_iter_meta *, meta); + __bpf_md_ptr(struct io_ring_ctx *, ctx); + __bpf_md_ptr(struct io_mapped_ubuf *, ubuf); + unsigned long index; +}; + +static void *__bpf_io_uring_buf_seq_get_next(struct bpf_io_uring_seq_info *info) +{ + if (info->index < info->ctx->nr_user_bufs) + return info->ctx->user_bufs[info->index++]; + return NULL; +} + +static void *bpf_io_uring_buf_seq_start(struct seq_file *seq, loff_t *pos) +{ + struct bpf_io_uring_seq_info *info = seq->private; + struct io_mapped_ubuf *ubuf; + + /* Indicate to userspace that the uring lock is contended */ + if (!mutex_trylock(&info->ctx->uring_lock)) + return ERR_PTR(-EDEADLK); + + ubuf = __bpf_io_uring_buf_seq_get_next(info); + if (!ubuf) + return NULL; + + if (*pos == 0) + ++*pos; + return ubuf; +} + +static void *bpf_io_uring_buf_seq_next(struct seq_file *seq, void *v, loff_t *pos) +{ + struct bpf_io_uring_seq_info *info = seq->private; + + ++*pos; + return __bpf_io_uring_buf_seq_get_next(info); +} + +DEFINE_BPF_ITER_FUNC(io_uring_buf, struct bpf_iter_meta *meta, + struct io_ring_ctx *ctx, struct io_mapped_ubuf *ubuf, + unsigned long index) + +static int __bpf_io_uring_buf_seq_show(struct seq_file *seq, void *v, bool in_stop) +{ + struct bpf_io_uring_seq_info *info = seq->private; + struct bpf_iter__io_uring_buf ctx; + struct bpf_iter_meta meta; + struct bpf_prog *prog; + + meta.seq = seq; + prog = bpf_iter_get_info(&meta, in_stop); + if (!prog) + return 0; + + ctx.meta = &meta; + ctx.ctx = info->ctx; + ctx.ubuf = v; + ctx.index = info->index ? info->index - !in_stop : 0; + + return bpf_iter_run_prog(prog, &ctx); +} + +static int bpf_io_uring_buf_seq_show(struct seq_file *seq, void *v) +{ + return __bpf_io_uring_buf_seq_show(seq, v, false); +} + +static void bpf_io_uring_buf_seq_stop(struct seq_file *seq, void *v) +{ + struct bpf_io_uring_seq_info *info = seq->private; + + /* If IS_ERR(v) is true, then ctx->uring_lock wasn't taken */ + if (IS_ERR(v)) + return; + if (!v) + __bpf_io_uring_buf_seq_show(seq, v, true); + else if (info->index) /* restart from index */ + info->index--; + mutex_unlock(&info->ctx->uring_lock); +} + +static const struct seq_operations bpf_io_uring_buf_seq_ops = { + .start = bpf_io_uring_buf_seq_start, + .next = bpf_io_uring_buf_seq_next, + .stop = bpf_io_uring_buf_seq_stop, + .show = bpf_io_uring_buf_seq_show, +}; + +static const struct bpf_iter_seq_info bpf_io_uring_buf_seq_info = { + .seq_ops = &bpf_io_uring_buf_seq_ops, + .init_seq_private = bpf_io_uring_init_seq, + .fini_seq_private = NULL, + .seq_priv_size = sizeof(struct bpf_io_uring_seq_info), +}; + +static struct bpf_iter_reg io_uring_buf_reg_info = { + .target = "io_uring_buf", + .feature = BPF_ITER_RESCHED, + .attach_target = bpf_io_uring_iter_attach, + .detach_target = bpf_io_uring_iter_detach, + .ctx_arg_info_size = 2, + .ctx_arg_info = { + { offsetof(struct bpf_iter__io_uring_buf, ctx), + PTR_TO_BTF_ID }, + { offsetof(struct bpf_iter__io_uring_buf, ubuf), + PTR_TO_BTF_ID_OR_NULL }, + }, + .seq_info = &bpf_io_uring_buf_seq_info, +}; + +static int __init io_uring_iter_init(void) +{ + io_uring_buf_reg_info.ctx_arg_info[0].btf_id = btf_io_uring_ids[0]; + io_uring_buf_reg_info.ctx_arg_info[1].btf_id = btf_io_uring_ids[1]; + return bpf_iter_reg_target(&io_uring_buf_reg_info); +} +late_initcall(io_uring_iter_init); + +#endif diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 56098c866704..ddb9d4520a3f 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1509,8 +1509,10 @@ int bpf_obj_get_user(const char __user *pathname, int flags); extern int bpf_iter_ ## target(args); \ int __init bpf_iter_ ## target(args) { return 0; } +struct io_ring_ctx; struct bpf_iter_aux_info { struct bpf_map *map; + struct io_ring_ctx *ctx; }; typedef int (*bpf_iter_attach_target_t)(struct bpf_prog *prog, diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 6297eafdc40f..3323defa99a1 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -91,6 +91,9 @@ union bpf_iter_link_info { struct { __u32 map_fd; } map; + struct { + __u32 io_uring_fd; + } io_uring; }; /* BPF syscall commands, see bpf(2) man-page for more details. */ diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 6297eafdc40f..3323defa99a1 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -91,6 +91,9 @@ union bpf_iter_link_info { struct { __u32 map_fd; } map; + struct { + __u32 io_uring_fd; + } io_uring; }; /* BPF syscall commands, see bpf(2) man-page for more details. */ From patchwork Tue Nov 16 05:42:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12621419 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DE7AC433F5 for ; Tue, 16 Nov 2021 05:58:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5D90860F38 for ; Tue, 16 Nov 2021 05:58:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238726AbhKPGBv (ORCPT ); Tue, 16 Nov 2021 01:01:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35388 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240161AbhKPGBm (ORCPT ); Tue, 16 Nov 2021 01:01:42 -0500 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64953C061231; Mon, 15 Nov 2021 21:42:46 -0800 (PST) Received: by mail-pl1-x642.google.com with SMTP id m24so6464471pls.10; Mon, 15 Nov 2021 21:42:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=u0lClWPrvD/K1h3LgglE1rPein98w8QPLX7xeqet6rg=; b=nIqUjdyXfJIJlTHhj+Wzyv5kOeWFHiXSJZu1cjHSdZG33TPnsO5MB5aE1C4JRgbtaA mmhKCAE2v4vfeZS8dZyaB3lwRc023VndgcC/ExPox3+4v3mjg8dHhbARHl1NYWFYfOY3 JkBfc1Yk/ZgIMZocQUQnP+XsrCr3KO1PZKwp1j6PrDgRO2dDh3zhcwJoTZwI5uNBK/MH xgxszz7O26rszx8vfFXErmfnomFI0sDoPR8U+EEv43b+mccAYfqTuEqPJknnqKqUotO3 yPdZqUlX1lOc0cf5SPZjgTYVd0TWGOHfMYtu4YCIJG56ntaOU6JFOt44q69q7Y3kAB7I 73qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=u0lClWPrvD/K1h3LgglE1rPein98w8QPLX7xeqet6rg=; b=A5MSi2YKDzd8AjitE2dJKq+G1l2y3zeFE2p8rzjrDs6DmktTjl9UikjKzZp/5EGNO7 o5gQYLL6Aq4SxcI6DH7PbH5EhvHnWqLb8HmoXbP6lPBbbDWbkUjZ65rJsZo2Z7u26Ym5 Nn67jenQ1ReJ5gy4rwgiuFjfAXgPiKyT2+a1gUHa3hw8sgC303hgn/ITLqVKX6rmL1iT h5cpmq2FBS64GOzGGB68NNEU/rEcJWTuTiYSuibKUvq4495JaTUdeliV5PO0sZF6HS33 bBv1A3OC3PBlb+lIPmhS6Xzc1zz/276b9pUQMxb6cxCQdZgT2AYF3uEhfC1c4bIZBCyB 1gkQ== X-Gm-Message-State: AOAM530DMoPu42PQVM7VC0HU2xXhRnk0sGJshVqjLz4xynjFSBkqoFKD 405zlVmGP60ANXx8Nq9EnbDIjn7FmVw= X-Google-Smtp-Source: ABdhPJxBGz/SRjgi7l0L6Q8eXVANhWP0NXj0Z57F/L2e+Rs3ReTGP+o5KTksilGOD6ZaEh1ojs4kUg== X-Received: by 2002:a17:90a:4b47:: with SMTP id o7mr35592429pjl.92.1637041365620; Mon, 15 Nov 2021 21:42:45 -0800 (PST) Received: from localhost ([2405:201:6014:d064:3d4e:6265:800c:dc84]) by smtp.gmail.com with ESMTPSA id ne22sm1002573pjb.18.2021.11.15.21.42.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 21:42:45 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Pavel Emelyanov , Alexander Mihalicyn , Andrei Vagin , criu@openvz.org, io-uring@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH bpf-next v1 2/8] bpf: Add bpf_page_to_pfn helper Date: Tue, 16 Nov 2021 11:12:31 +0530 Message-Id: <20211116054237.100814-3-memxor@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211116054237.100814-1-memxor@gmail.com> References: <20211116054237.100814-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5296; h=from:subject; bh=vnwFxGdaJpBG++VBChgGlEBqIAnV9UjRswVX4pSbVSY=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhk0S6qXxy7qrRuQKRRKAIPrBzMZcEaYxS3VShsUn5 OQUSrtCJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYZNEugAKCRBM4MiGSL8RyhmCEA CscqOFYalotS2DTRoBPvy+r8xLyKJ9AzMEnwRUfcK8OrSzIJn1n8mjRyGwBLnkShbC94pmq54fcxXx 0WJDgl0PUrIhxw2FIDbSOTI+RciU1hO9DSY6cH0dCa5JF6WpDBim2rDb+4UUs2tR9f0TOGKTUvummc DBBLKcgdrMbBh4fedbPK5UgB8elzM+BNFyvOgrkOfCBiiUjI312Y0qrM8TxK9XJjrsGQp90KTPwMP8 C2eg2HRa0yhmTwpCXqJWAYTz2yAPWyM/EYhV1nZfTqFM0zcc/CIqCjwJtV0d7Ccj1UzxIGa3lq9HoP q0X2aQPvQOO29lW/t+yf0sRsNE+VKqGkUAcAQAtdfQ3oL2+IUXmYT3LfArA4zrv30TpRX+M0sJXCOi v3L2MWELIX6XB00e+waSAi9dN1WU02dCILGZBpkQJm3Q70XrZRHSmT5DlXEc6ZaV5lcpySztIE9A7G mgc1FVeMk+mEVmTSRgiUtDJ5RhR8tEeN9YCwFqA8VYCUq0wA4Uort0i/mWa2KZmQOt7UJwjB8LsgH8 pFioVZsNj3J8uDiFf8LtR5F4nISevNB3Jn3/fR3DvLgmOQrnbOOFqNib6tlrNuuhUN1h5Mtig38IiB MgQLSNT6evdzBFcBY4rEsGQ+UcpQ2ZMocvXot1b0sszN4WPKQsr3NUPzryIA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org In CRIU, we need to be able to determine whether the page pinned by io_uring is still present in the same range in the process VMA. /proc//pagemap gives us the PFN, hence using this helper we can establish this mapping easily from the iterator side. It is a simple wrapper over the in-kernel page_to_pfn helper, and ensures the passed in pointer is a struct page PTR_TO_BTF_ID. This is obtained from the bvec of io_uring_ubuf for the CRIU usecase. Signed-off-by: Kumar Kartikeya Dwivedi Reported-by: kernel test robot Reported-by: kernel test robot --- fs/io_uring.c | 17 +++++++++++++++++ include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 9 +++++++++ kernel/trace/bpf_trace.c | 2 ++ scripts/bpf_doc.py | 2 ++ tools/include/uapi/linux/bpf.h | 9 +++++++++ 6 files changed, 40 insertions(+) diff --git a/fs/io_uring.c b/fs/io_uring.c index 46a110989155..9e9df6767e29 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -11295,6 +11295,23 @@ static struct bpf_iter_reg io_uring_buf_reg_info = { .seq_info = &bpf_io_uring_buf_seq_info, }; +BPF_CALL_1(bpf_page_to_pfn, struct page *, page) +{ + /* PTR_TO_BTF_ID can be NULL */ + if (!page) + return U64_MAX; + return page_to_pfn(page); +} + +BTF_ID_LIST_SINGLE(btf_page_to_pfn_ids, struct, page) + +const struct bpf_func_proto bpf_page_to_pfn_proto = { + .func = bpf_page_to_pfn, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_BTF_ID, + .arg1_btf_id = &btf_page_to_pfn_ids[0], +}; + static int __init io_uring_iter_init(void) { io_uring_buf_reg_info.ctx_arg_info[0].btf_id = btf_io_uring_ids[0]; diff --git a/include/linux/bpf.h b/include/linux/bpf.h index ddb9d4520a3f..fe7b499da781 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -2166,6 +2166,7 @@ extern const struct bpf_func_proto bpf_sk_setsockopt_proto; extern const struct bpf_func_proto bpf_sk_getsockopt_proto; extern const struct bpf_func_proto bpf_kallsyms_lookup_name_proto; extern const struct bpf_func_proto bpf_find_vma_proto; +extern const struct bpf_func_proto bpf_page_to_pfn_proto; const struct bpf_func_proto *tracing_prog_func_proto( enum bpf_func_id func_id, const struct bpf_prog *prog); diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 3323defa99a1..b70e9da3d722 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -4960,6 +4960,14 @@ union bpf_attr { * **-ENOENT** if *task->mm* is NULL, or no vma contains *addr*. * **-EBUSY** if failed to try lock mmap_lock. * **-EINVAL** for invalid **flags**. + * + * long bpf_page_to_pfn(struct page *page) + * Description + * Obtain the page frame number (PFN) for the given *struct page* + * pointer. + * Return + * Page Frame Number corresponding to the page pointed to by the + * *struct page* pointer, or U64_MAX if pointer is NULL. */ #define __BPF_FUNC_MAPPER(FN) \ FN(unspec), \ @@ -5143,6 +5151,7 @@ union bpf_attr { FN(skc_to_unix_sock), \ FN(kallsyms_lookup_name), \ FN(find_vma), \ + FN(page_to_pfn), \ /* */ /* integer value in 'imm' field of BPF_CALL instruction selects which helper diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 25ea521fb8f1..f68a8433be1a 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1212,6 +1212,8 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) return &bpf_find_vma_proto; case BPF_FUNC_trace_vprintk: return bpf_get_trace_vprintk_proto(); + case BPF_FUNC_page_to_pfn: + return &bpf_page_to_pfn_proto; default: return bpf_base_func_proto(func_id); } diff --git a/scripts/bpf_doc.py b/scripts/bpf_doc.py index a6403ddf5de7..ae68ca794980 100755 --- a/scripts/bpf_doc.py +++ b/scripts/bpf_doc.py @@ -549,6 +549,7 @@ class PrinterHelpers(Printer): 'struct socket', 'struct file', 'struct bpf_timer', + 'struct page', ] known_types = { '...', @@ -598,6 +599,7 @@ class PrinterHelpers(Printer): 'struct socket', 'struct file', 'struct bpf_timer', + 'struct page', } mapped_types = { 'u8': '__u8', diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 3323defa99a1..b70e9da3d722 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -4960,6 +4960,14 @@ union bpf_attr { * **-ENOENT** if *task->mm* is NULL, or no vma contains *addr*. * **-EBUSY** if failed to try lock mmap_lock. * **-EINVAL** for invalid **flags**. + * + * long bpf_page_to_pfn(struct page *page) + * Description + * Obtain the page frame number (PFN) for the given *struct page* + * pointer. + * Return + * Page Frame Number corresponding to the page pointed to by the + * *struct page* pointer, or U64_MAX if pointer is NULL. */ #define __BPF_FUNC_MAPPER(FN) \ FN(unspec), \ @@ -5143,6 +5151,7 @@ union bpf_attr { FN(skc_to_unix_sock), \ FN(kallsyms_lookup_name), \ FN(find_vma), \ + FN(page_to_pfn), \ /* */ /* integer value in 'imm' field of BPF_CALL instruction selects which helper From patchwork Tue Nov 16 05:42:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12621423 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92A72C433EF for ; Tue, 16 Nov 2021 05:59:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7B52F63218 for ; Tue, 16 Nov 2021 05:59:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239668AbhKPGCD (ORCPT ); Tue, 16 Nov 2021 01:02:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35350 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240118AbhKPGBm (ORCPT ); Tue, 16 Nov 2021 01:01:42 -0500 Received: from mail-pl1-x644.google.com (mail-pl1-x644.google.com [IPv6:2607:f8b0:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30BBEC0611FE; Mon, 15 Nov 2021 21:42:49 -0800 (PST) Received: by mail-pl1-x644.google.com with SMTP id y7so16552769plp.0; Mon, 15 Nov 2021 21:42:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9m5mY4Ir11w/zrxUgoXRmOQA6YWYJV+7fBqTFmvYDJ8=; b=Gk52qAmukeILt+Z85X6dzczGbu79c1p3NieBJNOO34A91qDODEMW0Mr/y3WPMl2Uw7 kS2uCDzj8GXhqDSALm9w523gGn0kaVUhc6YgyMKZSgeYA+4caUHA4F6Wjv4eYw9PyYYs E3qEahuq5GmUTSBFLvygBRFvZxujKgsDlAsu7W00AFIcayKI6BFRvECIbY9MedCUJlrX NpDcPblOJK3KghUyz4OepUVvaKAmu26DgUJ45ZX+/huW4tebQETXLzPIBhBgqSs/+CPF G3j9lMGevalo8uIdhwu2/tCM+My+bpKIZhr/Ny9x0HVWIU3dPYRYntw41pkke3OHFQvT Txxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9m5mY4Ir11w/zrxUgoXRmOQA6YWYJV+7fBqTFmvYDJ8=; b=y3xX2EOqRGDp5D//DXX1GShhMhzD+tQOtztVAcNdBSVpXk3akTTBQzwhJsdvr7Z2Jv tuMqaVpjc/agqiNO6qOufKDVb9ZlDiYwGgItJfGX52vJOMoeMjCBxJKa8mW68zGCbKut 6sFLtIOI5mmAaXoXLSv70TXC5yamOT0UQrWLIUhlMJFJLcgXGqv1UlH9S9LlNrPjtniO WN233n8Ok5141fGzq4fkZQCA29XMWUEa9ayqQXI3hrHUk6xZT8F2VMPSXBjSTSBCzsuX BEc0IIegoDPXNAKlnkvei4DBhpKR69HGD3NIECGr5X9TfrJDau6tNCKDOxQ26GXwdITY cv6w== X-Gm-Message-State: AOAM533Ao1EvB+AqBoF0pdLskjyVON5WI+IgJgg2G2DdhkBaLOPo1mhf O1MGSz7QQLhcmc+uxp8PvcDKExIeDl8= X-Google-Smtp-Source: ABdhPJyAHF2SFmea7g28GZKoITtfRE8lZMMxjXe5h83cj6X7/sl/AnJIQaH7JdjBxqwNJ1Zzgx70Qw== X-Received: by 2002:a17:90a:5917:: with SMTP id k23mr72424906pji.111.1637041368569; Mon, 15 Nov 2021 21:42:48 -0800 (PST) Received: from localhost ([2405:201:6014:d064:3d4e:6265:800c:dc84]) by smtp.gmail.com with ESMTPSA id l28sm14027251pgu.45.2021.11.15.21.42.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 21:42:48 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Jens Axboe , Pavel Begunkov , io-uring@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Pavel Emelyanov , Alexander Mihalicyn , Andrei Vagin , criu@openvz.org, linux-fsdevel@vger.kernel.org Subject: [PATCH bpf-next v1 3/8] io_uring: Implement eBPF iterator for registered files Date: Tue, 16 Nov 2021 11:12:32 +0530 Message-Id: <20211116054237.100814-4-memxor@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211116054237.100814-1-memxor@gmail.com> References: <20211116054237.100814-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=7590; h=from:subject; bh=4UYxWbGiq/sKKUnwuIEWRtr0AZqHxKqyXPv77YH/sHg=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhk0S6lWQYf/gzn1knBEAdQjsAn3cbdKR+Up62zDhc W4k4mUuJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYZNEugAKCRBM4MiGSL8RytrpEA Cauy4Jmn2ObLRVsjUaSh0Iq1nJQpFOz43qV8XL0spPojOttz/VC0LwkE8nOlBAIbWpTTZfODq22tM7 /vVBSSsAVh69WB/gKECy94TLOzW9iOdPhc9PWDar7tgcDHRc1rdRBgADkh9c/6Ljf5lhfEX9KskJIH 7eQbFOkQHHtZ1Drv/YIXtLxqBH24xU0xvzjPrd3rlwkOr18V7coPfSyebSpFPs2EMpPjR/LsixWRJU IzS1IHF1dgxIYqXlB6qdvgSIndO84D84PYCCMWrgG3mKTUEAHDwbEWUmjvv2LjwNsxa1wPYo4g2I0+ /vS6+noI1aCcyOUyD06pfvoVEFSa5isUb5tG4+cxSoqkj4nv03+MtijLpe6DOJ8TGxxzdg//mMT78A SfDANOv6SH8yiivjKpntGEES3xV+Lu9ts30+yFCPpdRZnc4M1uE1y61tX/SMsviXo8opa4Of/V/A6c x/Xpo9US36ogzL9+KdHF11sQJM9MusJq6fVylPHd9V9xsfLsxvAcgjEinwusm8rpyOqAyL8EIXBJEr SpSApGu/nmWQZdXzyXGF+VMrKp6roh4hu6+K4APcLaeKG2ylNZB7a8q8DjBnZc3kyhMyP3FiE4ZC// YTBOvEztGUwYWmnodZpius+WwV7JCJqV6Xs4cFecSwHI71sk+VTHxJD+0+rQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This change adds eBPF iterator for buffers registered in io_uring ctx. It gives access to the ctx, the index of the registered buffer, and a pointer to the struct file itself. This allows the iterator to save info related to the file added to an io_uring instance, that isn't easy to export using the fdinfo interface (like being able to match registered files to a task's file set). Getting access to underlying struct file allows deduplication and efficient pairing with task file set (obtained using task_file iterator). The primary usecase this is enabling is checkpoint/restore support. Note that we need to use mutex_trylock when the file is read from, in seq_start functions, as the order of lock taken is opposite of what it would be when io_uring operation reads the same file. We take seq_file->lock, then ctx->uring_lock, while io_uring would first take ctx->uring_lock and then seq_file->lock for the same ctx. This can lead to a deadlock scenario described below: CPU 0 CPU 1 vfs_read mutex_lock(&seq_file->lock) io_read mutex_lock(&ctx->uring_lock) mutex_lock(&ctx->uring_lock) # switched to mutex_trylock mutex_lock(&seq_file->lock) The trylock also protects the case where io_uring tries to read from iterator attached to itself (same ctx), where the order of locks would be: io_uring_enter mutex_lock(&ctx->uring_lock) <-----------. io_read \ seq_read \ mutex_lock(&seq_file->lock) / mutex_lock(&ctx->uring_lock) # deadlock-` In both these cases (recursive read and contended uring_lock), -EDEADLK is returned to userspace. With the advent of descriptorless files supported by io_uring, this iterator provides the required visibility and introspection of io_uring instance for the purposes of dumping and restoring it. In the future, this iterator will be extended to support direct inspection of a lot of file state (currently descriptorless files are obtained using openat2 and socket) to dump file state for these hidden files. Later, we can explore filling in the gaps for dumping file state for more file types (those not hidden in io_uring ctx). All this is out of scope for the current series however, but builds upon this iterator. Cc: Jens Axboe Cc: Pavel Begunkov Cc: io-uring@vger.kernel.org Signed-off-by: Kumar Kartikeya Dwivedi --- fs/io_uring.c | 140 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 139 insertions(+), 1 deletion(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 9e9df6767e29..7ac479c95d4e 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -11132,6 +11132,7 @@ __initcall(io_uring_init); BTF_ID_LIST(btf_io_uring_ids) BTF_ID(struct, io_ring_ctx) BTF_ID(struct, io_mapped_ubuf) +BTF_ID(struct, file) struct bpf_io_uring_seq_info { struct io_ring_ctx *ctx; @@ -11312,11 +11313,148 @@ const struct bpf_func_proto bpf_page_to_pfn_proto = { .arg1_btf_id = &btf_page_to_pfn_ids[0], }; +/* io_uring iterator for registered files */ + +struct bpf_iter__io_uring_file { + __bpf_md_ptr(struct bpf_iter_meta *, meta); + __bpf_md_ptr(struct io_ring_ctx *, ctx); + __bpf_md_ptr(struct file *, file); + unsigned long index; +}; + +static void *__bpf_io_uring_file_seq_get_next(struct bpf_io_uring_seq_info *info) +{ + struct file *file = NULL; + + if (info->index < info->ctx->nr_user_files) { + /* file set can be sparse */ + file = io_file_from_index(info->ctx, info->index++); + /* use info as a distinct pointer to distinguish between empty + * slot and valid file, since we cannot return NULL for this + * case if we want iter prog to still be invoked with file == + * NULL. + */ + if (!file) + return info; + } + + return file; +} + +static void *bpf_io_uring_file_seq_start(struct seq_file *seq, loff_t *pos) +{ + struct bpf_io_uring_seq_info *info = seq->private; + struct file *file; + + /* Indicate to userspace that the uring lock is contended */ + if (!mutex_trylock(&info->ctx->uring_lock)) + return ERR_PTR(-EDEADLK); + + file = __bpf_io_uring_file_seq_get_next(info); + if (!file) + return NULL; + + if (*pos == 0) + ++*pos; + return file; +} + +static void *bpf_io_uring_file_seq_next(struct seq_file *seq, void *v, loff_t *pos) +{ + struct bpf_io_uring_seq_info *info = seq->private; + + ++*pos; + return __bpf_io_uring_file_seq_get_next(info); +} + +DEFINE_BPF_ITER_FUNC(io_uring_file, struct bpf_iter_meta *meta, + struct io_ring_ctx *ctx, struct file *file, + unsigned long index) + +static int __bpf_io_uring_file_seq_show(struct seq_file *seq, void *v, bool in_stop) +{ + struct bpf_io_uring_seq_info *info = seq->private; + struct bpf_iter__io_uring_file ctx; + struct bpf_iter_meta meta; + struct bpf_prog *prog; + + meta.seq = seq; + prog = bpf_iter_get_info(&meta, in_stop); + if (!prog) + return 0; + + ctx.meta = &meta; + ctx.ctx = info->ctx; + /* when we encounter empty slot, v will point to info */ + ctx.file = v == info ? NULL : v; + ctx.index = info->index ? info->index - !in_stop : 0; + + return bpf_iter_run_prog(prog, &ctx); +} + +static int bpf_io_uring_file_seq_show(struct seq_file *seq, void *v) +{ + return __bpf_io_uring_file_seq_show(seq, v, false); +} + +static void bpf_io_uring_file_seq_stop(struct seq_file *seq, void *v) +{ + struct bpf_io_uring_seq_info *info = seq->private; + + /* If IS_ERR(v) is true, then ctx->uring_lock wasn't taken */ + if (IS_ERR(v)) + return; + if (!v) + __bpf_io_uring_file_seq_show(seq, v, true); + else if (info->index) /* restart from index */ + info->index--; + mutex_unlock(&info->ctx->uring_lock); +} + +static const struct seq_operations bpf_io_uring_file_seq_ops = { + .start = bpf_io_uring_file_seq_start, + .next = bpf_io_uring_file_seq_next, + .stop = bpf_io_uring_file_seq_stop, + .show = bpf_io_uring_file_seq_show, +}; + +static const struct bpf_iter_seq_info bpf_io_uring_file_seq_info = { + .seq_ops = &bpf_io_uring_file_seq_ops, + .init_seq_private = bpf_io_uring_init_seq, + .fini_seq_private = NULL, + .seq_priv_size = sizeof(struct bpf_io_uring_seq_info), +}; + +static struct bpf_iter_reg io_uring_file_reg_info = { + .target = "io_uring_file", + .feature = BPF_ITER_RESCHED, + .attach_target = bpf_io_uring_iter_attach, + .detach_target = bpf_io_uring_iter_detach, + .ctx_arg_info_size = 2, + .ctx_arg_info = { + { offsetof(struct bpf_iter__io_uring_file, ctx), + PTR_TO_BTF_ID }, + { offsetof(struct bpf_iter__io_uring_file, file), + PTR_TO_BTF_ID_OR_NULL }, + }, + .seq_info = &bpf_io_uring_file_seq_info, +}; + static int __init io_uring_iter_init(void) { + int ret; + io_uring_buf_reg_info.ctx_arg_info[0].btf_id = btf_io_uring_ids[0]; io_uring_buf_reg_info.ctx_arg_info[1].btf_id = btf_io_uring_ids[1]; - return bpf_iter_reg_target(&io_uring_buf_reg_info); + io_uring_file_reg_info.ctx_arg_info[0].btf_id = btf_io_uring_ids[0]; + io_uring_file_reg_info.ctx_arg_info[1].btf_id = btf_io_uring_ids[2]; + ret = bpf_iter_reg_target(&io_uring_buf_reg_info); + if (ret) + return ret; + ret = bpf_iter_reg_target(&io_uring_file_reg_info); + if (ret) + bpf_iter_unreg_target(&io_uring_buf_reg_info); + return ret; } late_initcall(io_uring_iter_init); From patchwork Tue Nov 16 05:42:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12621425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67B84C433EF for ; Tue, 16 Nov 2021 05:59:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4EFD76321A for ; Tue, 16 Nov 2021 05:59:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239015AbhKPGCR (ORCPT ); Tue, 16 Nov 2021 01:02:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34624 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240231AbhKPGBn (ORCPT ); Tue, 16 Nov 2021 01:01:43 -0500 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 25C65C061193; Mon, 15 Nov 2021 21:42:52 -0800 (PST) Received: by mail-pj1-x1044.google.com with SMTP id np3so14799836pjb.4; Mon, 15 Nov 2021 21:42:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=P7YHBodN7FxxKr28EL14BHBK1Op3LrUSznhVoFpHxSg=; b=Vx0LsaymW/BH3nFK3ImCZEkLGDhG8b7cXMGwMdEt/3BsXgLEM/ehd5coQVVkrhleol Ct8UKkvePleY70M7373gHj5QBHKrEYpfZqT7eAe5akHV8gKcFC1mj++PSVTBnovN6fBk g2PXINM9lCyKHMGmayFQCDl+TTBOZybBIRIce842JDEHJzm1geHN5OMLugY3DqZ87QRJ jiroE+JVXtWnLQPFeJKoqEMD75bG298hLt+FfIdW2XNomdZOtMqHczE3P7P/yiuardPI e/co+FLqTIkxFcseJ162aKMUdS3qnMEsbZaskutqctnOpuePdqGJf5iBf8zefJQfG7iJ CgPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=P7YHBodN7FxxKr28EL14BHBK1Op3LrUSznhVoFpHxSg=; b=0I6dzjeksf0MLdyxnP3xssJiiCMap7jCIQQZZb7GefktydpGrxEwhDyryys3VL8oy2 9Ma808IFtTOKfKskMckYjybGbxYJshgmmb8R4TfDGOqLU0YneeXcymEDeftdC3nd5YIy KIS6FVnzKUkgAzGz40elWAxsGI2rRq8WGcGGaWaBI4txETeE9OfyFxYDEDTCRrCvHoVG ifNPHc0abC42uLrh8sNq6XveONgDDqvT43MLYcjOPZOhY9t724D75XkmwlEMYozY3uZ4 xcTIC5CkWcYIvQFeWSWSEGKIaDpxWsQT4W6OGtc7qFCdblczPV01BivLI78NGf9/jElf rFEg== X-Gm-Message-State: AOAM533WymGRabou/nJmBS0JY4AWWgOYAS6gAlIQtk28jkogz2n/gzay /B4bjfRBHPG60CJlnOSwkL6nrw0ain8= X-Google-Smtp-Source: ABdhPJwHG7ivrchw7+i00w5+bYy9YEmxelyGxTw88RyTufsUwJGM5cxq5mqK1o3nczYuzByK4A2dCA== X-Received: by 2002:a17:90b:1a87:: with SMTP id ng7mr5586200pjb.230.1637041371415; Mon, 15 Nov 2021 21:42:51 -0800 (PST) Received: from localhost ([2405:201:6014:d064:3d4e:6265:800c:dc84]) by smtp.gmail.com with ESMTPSA id j9sm12439968pgt.54.2021.11.15.21.42.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 21:42:51 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexander Viro , linux-fsdevel@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Pavel Emelyanov , Alexander Mihalicyn , Andrei Vagin , criu@openvz.org, io-uring@vger.kernel.org Subject: [PATCH bpf-next v1 4/8] epoll: Implement eBPF iterator for registered items Date: Tue, 16 Nov 2021 11:12:33 +0530 Message-Id: <20211116054237.100814-5-memxor@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211116054237.100814-1-memxor@gmail.com> References: <20211116054237.100814-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=10782; h=from:subject; bh=1NPE/ek3S/8oo41vaRgKFoZuG2Bc9SehhcdLlVfzD04=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhk0S6HKYlvIkVLhiUHzQs1cWeiY6nMqQDST3wOv1f cZobjnqJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYZNEugAKCRBM4MiGSL8RyqiDEA CZ7s3gkblUatbO0TS+0BeayG91r+qANpUFSI88DULe2yOxz19QhLaV6zXNIk2z0HgmmKZYsAzZvLSg f4j2pBe7xHDMA0d1g9jziLX3zRiEhaLzRPAqFz9JlhVBmmlxOPPYxQMHo0Z8FGnwDy8M73XvSyTHEs rfX9AGGeLe2guIhhilv3GWsSLUP4pa3uzmLmZxslIFrSSnAPgKFcj2pfwdA1mazFtUISyJwuhaubgv tAgqdh7GUSMfcrpmeiX4KYs/2kYOrv80ZpMD1iuGSIsaixAjZpvc230Tjn9qFSdTdO60Ijq9uTXMwf oLvczNrmgS949KfxPFSt4lhZGZp7EOHB+u4XVf/XE4iHQtNQWs+6HAa1bgy60ferszi+izQxdV7qSx VsHCmtzqWKHkQ54aAXZUwEqZTYLqHKDpKgnKy282yDyGzSdhVccfa2T9dlwkwau460tQYoRAiBeuzo kNOCH3/UUA0qZA+wJoxjxMhZTyLxwSEZldhfY0hZqPw++4DpYKa9hMI/SC8tzRDOEgeYW++OuYrJuc r3lKBF+L4Y4Q/6rNSUb5OPP58UeeLoSOAdOtSvOBARmT2rAvTz7AormxdjSctpry1a0YpJgESt7C7m A30AObmfiqIMaMKzUPmUf4yl4lyqmL8Vbe2Jr80Yzvrur0LyzaXGe/ZJt0dg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This patch adds eBPF iterator for epoll items (epitems) registered in an epoll instance. It gives access to the eventpoll ctx, and the registered epoll item (struct epitem). This allows the iterator to inspect the registered file and be able to use others iterators to associate it with a task's fdtable. The primary usecase this is enabling is expediting existing eventpoll checkpoint/restore support in the CRIU project. This iterator allows us to switch from a worst case O(n^2) algorithm to a single O(n) pass over task and epoll registered descriptors. We also make sure we're iterating over a live file, one that is not going away. The case we're concerned about is a file that has its f_count as zero, but is waiting for iterator bpf_seq_read to release ep->mtx, so that it can remove its epitem. Since such a file will disappear once iteration is done, and it is being destructed, we use get_file_rcu to ensure it is alive when invoking the BPF program. Getting access to a file that is going to disappear after iteration is not useful anyway. This does have a performance overhead however (since file reference will be raised and dropped for each file). The rcu_read_lock around get_file_rcu isn't strictly required for lifetime management since fput path is serialized on ep->mtx to call ep_remove, hence the epi->ffd.file pointer remains stable during our seq_start/seq_stop bracketing. To be able to continue back from the position we were iterating, we store the epi->ffi.fd and use ep_find_tfd to find the target file again. It would be more appropriate to use both struct file pointer and fd number to find the last file, but see below for why that cannot be done. Taking reference to struct file and walking RB-Tree to find it again will lead to reference cycle issue if the iterator after partial read takes reference to socket which later is used in creating a descriptor cycle using SCM_RIGHTS. An example that was encountered when working on this is mentioned below. Let there be Unix sockets SK1, SK2, epoll fd EP, and epoll iterator ITER. Let SK1 be registered in EP, then on a partial read it is possible that ITER returns from read and takes reference to SK1 to be able to find it later in RB-Tree and continue the iteration. If SK1 sends ITER over to SK2 using SCM_RIGHTS, and SK2 sends SK2 over to SK1 using SCM_RIGHTS, and both fds are not consumed on the corresponding receive ends, a cycle is created. When all of SK1, SK2, EP, and ITER are closed, SK1's receive queue holds reference to SK2, and SK2's receive queue holds reference to ITER, which holds a reference to SK1. All file descriptors except EP leak. To resolve it, we would need to hook into the Unix Socket GC mechanism, but the alternative of using ep_find_tfd is much more simpler. The finding of the last position in face of concurrent modification of the epoll set is at best an approximation anyway. For the case of CRIU, the epoll set remains stable. Cc: Alexander Viro Cc: linux-fsdevel@vger.kernel.org Signed-off-by: Kumar Kartikeya Dwivedi --- fs/eventpoll.c | 196 ++++++++++++++++++++++++++++++++- include/linux/bpf.h | 5 +- include/uapi/linux/bpf.h | 3 + tools/include/uapi/linux/bpf.h | 3 + 4 files changed, 205 insertions(+), 2 deletions(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 06f4c5ae1451..aa21628b6307 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -37,6 +37,7 @@ #include #include #include +#include #include /* @@ -985,7 +986,6 @@ static struct epitem *ep_find(struct eventpoll *ep, struct file *file, int fd) return epir; } -#ifdef CONFIG_KCMP static struct epitem *ep_find_tfd(struct eventpoll *ep, int tfd, unsigned long toff) { struct rb_node *rbp; @@ -1005,6 +1005,7 @@ static struct epitem *ep_find_tfd(struct eventpoll *ep, int tfd, unsigned long t return NULL; } +#ifdef CONFIG_KCMP struct file *get_epoll_tfile_raw_ptr(struct file *file, int tfd, unsigned long toff) { @@ -2385,3 +2386,196 @@ static int __init eventpoll_init(void) return 0; } fs_initcall(eventpoll_init); + +#ifdef CONFIG_BPF_SYSCALL + +BTF_ID_LIST(btf_epoll_ids) +BTF_ID(struct, eventpoll) +BTF_ID(struct, epitem) + +struct bpf_epoll_iter_seq_info { + struct eventpoll *ep; + struct rb_node *rbp; + int tfd; +}; + +static int bpf_epoll_init_seq(void *priv_data, struct bpf_iter_aux_info *aux) +{ + struct bpf_epoll_iter_seq_info *info = priv_data; + + info->ep = aux->ep->private_data; + info->tfd = -1; + return 0; +} + +static int bpf_epoll_iter_attach(struct bpf_prog *prog, + union bpf_iter_link_info *linfo, + struct bpf_iter_aux_info *aux) +{ + struct file *file; + int ret; + + file = fget(linfo->epoll.epoll_fd); + if (!file) + return -EBADF; + + ret = -EOPNOTSUPP; + if (unlikely(!is_file_epoll(file))) + goto out_fput; + + aux->ep = file; + return 0; +out_fput: + fput(file); + return ret; +} + +static void bpf_epoll_iter_detach(struct bpf_iter_aux_info *aux) +{ + fput(aux->ep); +} + +struct bpf_iter__epoll { + __bpf_md_ptr(struct bpf_iter_meta *, meta); + __bpf_md_ptr(struct eventpoll *, ep); + __bpf_md_ptr(struct epitem *, epi); +}; + +static void *bpf_epoll_seq_start(struct seq_file *seq, loff_t *pos) +{ + struct bpf_epoll_iter_seq_info *info = seq->private; + struct epitem *epi; + + mutex_lock(&info->ep->mtx); + /* already iterated? */ + if (info->tfd == -2) + return NULL; + /* partially iterated */ + if (info->tfd >= 0) { + epi = ep_find_tfd(info->ep, info->tfd, 0); + if (!epi) + return NULL; + info->rbp = &epi->rbn; + return epi; + } + WARN_ON(info->tfd != -1); + /* first iteration */ + info->rbp = rb_first_cached(&info->ep->rbr); + if (!info->rbp) + return NULL; + if (*pos == 0) + ++*pos; + return rb_entry(info->rbp, struct epitem, rbn); +} + +static void *bpf_epoll_seq_next(struct seq_file *seq, void *v, loff_t *pos) +{ + struct bpf_epoll_iter_seq_info *info = seq->private; + + ++*pos; + info->rbp = rb_next(info->rbp); + return info->rbp ? rb_entry(info->rbp, struct epitem, rbn) : NULL; +} + +DEFINE_BPF_ITER_FUNC(epoll, struct bpf_iter_meta *meta, struct eventpoll *ep, + struct epitem *epi) + +static int __bpf_epoll_seq_show(struct seq_file *seq, void *v, bool in_stop) +{ + struct bpf_epoll_iter_seq_info *info = seq->private; + struct bpf_iter__epoll ctx; + struct bpf_iter_meta meta; + struct bpf_prog *prog; + int ret; + + meta.seq = seq; + prog = bpf_iter_get_info(&meta, in_stop); + if (!prog) + return 0; + + ctx.meta = &meta; + ctx.ep = info->ep; + ctx.epi = v; + if (ctx.epi) { + /* The file we are going to pass to prog may already have its f_count as + * 0, hence before invoking the prog, we always try to get the reference + * if it isn't zero, failing which we skip the file. This is usually the + * case for files that are closed before calling EPOLL_CTL_DEL for them, + * which would wait for us to release ep->mtx before doing ep_remove. + */ + rcu_read_lock(); + ret = get_file_rcu(ctx.epi->ffd.file); + rcu_read_unlock(); + if (!ret) + return 0; + } + ret = bpf_iter_run_prog(prog, &ctx); + /* fput queues work asynchronously, so in our case, either task_work + * for non-exiting task, and otherwise delayed_fput, so holding + * ep->mtx and calling fput (which will take the same lock) in + * this context will not deadlock us, in case f_count is 1 at this + * point. + */ + if (ctx.epi) + fput(ctx.epi->ffd.file); + return ret; +} + +static int bpf_epoll_seq_show(struct seq_file *seq, void *v) +{ + return __bpf_epoll_seq_show(seq, v, false); +} + +static void bpf_epoll_seq_stop(struct seq_file *seq, void *v) +{ + struct bpf_epoll_iter_seq_info *info = seq->private; + struct epitem *epi; + + if (!v) { + __bpf_epoll_seq_show(seq, v, true); + /* done iterating */ + info->tfd = -2; + } else { + epi = rb_entry(info->rbp, struct epitem, rbn); + info->tfd = epi->ffd.fd; + } + mutex_unlock(&info->ep->mtx); +} + +static const struct seq_operations bpf_epoll_seq_ops = { + .start = bpf_epoll_seq_start, + .next = bpf_epoll_seq_next, + .stop = bpf_epoll_seq_stop, + .show = bpf_epoll_seq_show, +}; + +static const struct bpf_iter_seq_info bpf_epoll_seq_info = { + .seq_ops = &bpf_epoll_seq_ops, + .init_seq_private = bpf_epoll_init_seq, + .seq_priv_size = sizeof(struct bpf_epoll_iter_seq_info), +}; + +static struct bpf_iter_reg epoll_reg_info = { + .target = "epoll", + .feature = BPF_ITER_RESCHED, + .attach_target = bpf_epoll_iter_attach, + .detach_target = bpf_epoll_iter_detach, + .ctx_arg_info_size = 2, + .ctx_arg_info = { + { offsetof(struct bpf_iter__epoll, ep), + PTR_TO_BTF_ID }, + { offsetof(struct bpf_iter__epoll, epi), + PTR_TO_BTF_ID_OR_NULL }, + }, + .seq_info = &bpf_epoll_seq_info, +}; + +static int __init epoll_iter_init(void) +{ + epoll_reg_info.ctx_arg_info[0].btf_id = btf_epoll_ids[0]; + epoll_reg_info.ctx_arg_info[1].btf_id = btf_epoll_ids[1]; + return bpf_iter_reg_target(&epoll_reg_info); +} +late_initcall(epoll_iter_init); + +#endif diff --git a/include/linux/bpf.h b/include/linux/bpf.h index fe7b499da781..eb1c9acdc40b 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1512,7 +1512,10 @@ int bpf_obj_get_user(const char __user *pathname, int flags); struct io_ring_ctx; struct bpf_iter_aux_info { struct bpf_map *map; - struct io_ring_ctx *ctx; + union { + struct io_ring_ctx *ctx; + struct file *ep; + }; }; typedef int (*bpf_iter_attach_target_t)(struct bpf_prog *prog, diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index b70e9da3d722..64e18c1dcfca 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -94,6 +94,9 @@ union bpf_iter_link_info { struct { __u32 io_uring_fd; } io_uring; + struct { + __u32 epoll_fd; + } epoll; }; /* BPF syscall commands, see bpf(2) man-page for more details. */ diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index b70e9da3d722..64e18c1dcfca 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -94,6 +94,9 @@ union bpf_iter_link_info { struct { __u32 io_uring_fd; } io_uring; + struct { + __u32 epoll_fd; + } epoll; }; /* BPF syscall commands, see bpf(2) man-page for more details. */ From patchwork Tue Nov 16 05:42:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12621427 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07906C433F5 for ; Tue, 16 Nov 2021 05:59:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D35C461C14 for ; Tue, 16 Nov 2021 05:59:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239427AbhKPGCX (ORCPT ); Tue, 16 Nov 2021 01:02:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35604 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240227AbhKPGBn (ORCPT ); Tue, 16 Nov 2021 01:01:43 -0500 Received: from mail-pf1-x443.google.com (mail-pf1-x443.google.com [IPv6:2607:f8b0:4864:20::443]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1172DC061195; Mon, 15 Nov 2021 21:42:55 -0800 (PST) Received: by mail-pf1-x443.google.com with SMTP id n85so17091529pfd.10; Mon, 15 Nov 2021 21:42:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NxCD8KaiHPLXnkvOJl2UV/cBQmsXSELbPufgXpoN6c4=; b=dj9LMKUBAI720gAvYXAxTU9ey2qecgjPnCqd7O+HcwRhM8PTLCjGoUsT6eenbFthSf 2i9koZE9d+bTFhaf56Gyjc3rxtMTz7/cZS6BO48SQ+1EwQWNITAK3EQtccoDiwG8gofX pZ6dW+s2/9AMUodaZn8D1iXUrLMLY/qRtG7jIQjSMwLCFJMCGSzef/wL6whe1If+1eHn 6QU2XFIF9feLcGXR3JFDab5kKhMul3U2KDUEbMU+XO7zb1x3rA7eYHiMdCFmeHkcO0ta DR+8mZvXwVCHCuZWbl2wxyroM0asZhktF0CENiTN+hom45hTC4xE5uWWuxCm2cCv7kdN BkIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NxCD8KaiHPLXnkvOJl2UV/cBQmsXSELbPufgXpoN6c4=; b=Z+qZNcFp7Stpa0R7owvIci/cvnSlMwz3ENFmPAAS65z3/WHHtYUCgLjj8R0RSyvwB5 BgC3WzKAwnrVwqafkHZLQ8Vm1/T+YWZpbWnqjaJs5TvWdIB/GMJwoqDF7yvXFLq1XvDT EZ7/kluxNUcJKKwnvzMXC7yC7+5SweuFWqmiU4WvUwvPOWZaeaFZ4MdokB9hv/2rjOSR n2/vj/kUKias4EvvKRVT5iEm4+SanZn1XQEIYFx2WBWrzsmTYTeKGlg2F2iBdHezK9q8 nbCQPSspMOMQCdiVOAL15C1wvjP404fyU9AOLTg6gjwHzHokbjc7TSDODcDrsswyDSun wSuw== X-Gm-Message-State: AOAM532NFBXQAMoTdJvIQbsOQ2vHuxkvGhk0OKKoQ+2t85BCsWAgAJlq nFxwxUusXoIe6RuGkIHjxw1xI2FixoE= X-Google-Smtp-Source: ABdhPJxq7PocpHAbxYMQqbVhz9RI9mAGzp8d9bN7B6WY3qIBeeqxcCGdlHxXoUXHNeVMiehBqxJjNw== X-Received: by 2002:a05:6a00:a8e:b0:47b:a658:7f4d with SMTP id b14-20020a056a000a8e00b0047ba6587f4dmr37797412pfl.82.1637041374376; Mon, 15 Nov 2021 21:42:54 -0800 (PST) Received: from localhost ([2405:201:6014:d064:3d4e:6265:800c:dc84]) by smtp.gmail.com with ESMTPSA id h196sm291525pfe.216.2021.11.15.21.42.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 21:42:54 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Jens Axboe , Pavel Begunkov , io-uring@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Pavel Emelyanov , Alexander Mihalicyn , Andrei Vagin , criu@openvz.org, linux-fsdevel@vger.kernel.org Subject: [PATCH bpf-next v1 5/8] selftests/bpf: Add test for io_uring BPF iterators Date: Tue, 16 Nov 2021 11:12:34 +0530 Message-Id: <20211116054237.100814-6-memxor@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211116054237.100814-1-memxor@gmail.com> References: <20211116054237.100814-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=10072; h=from:subject; bh=eMb/lQU8qKhODY9ILUiWixTNYnj2WoiakRkQyzvXpRU=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhk0S6UV/NukBUm77oBDytNGrgkMltKKtqdGy4l9R8 GNuM0t2JAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYZNEugAKCRBM4MiGSL8RyszpEA C8X5u0HNoe4RDIwcWxn9hoMz6pZnl2Wk5f3DB/QI3sa4PtG7whXr3I5/bET9/2cHGjc0bksOhTZSwl JUpPoorJHKvarTxMhWqEdPStXkvd8iVHlOOQ+t97jq5Jyn8UqNsWAPrga8Ik2YpSnE64r0bZU+Chz+ uBf1sf7N9fC3hPzIifHsjZCkY6bSCDp9tQPqmkD931JP2pOUsDlOkfODfqn7WPhDzxxFinQFGKWPlB Y/26EA9kC1ZJEeF9IYQ4omFJCfYPRiNdbGEMljf8CxKxEW0T2at/vOVT+9vT+pP4UBGnsKdU+mQpZd HpZssIIqZCZFzCRUJ6z5AAMMmWadNE8k0fsXwV1fW5xu5OMfTEzW5NZx9U3EfbFwnZ1Aq9Co3B7wY2 1Bx5Byvv0XRfaHKq3QI9MrdtDYAY6wb8TCy72XTIz5yhaTBgeLQ2SEyd3fFp8XXXGpS7jTZFmAbeiA 3z5GVPya0KDasYB67DmTTpptermcCQiOEd8sQ0BY6f1071HDoWFhih8yhntL8bFou7it1YADEQR6RA OEKGTChbSpMXuyKnkPm2289yAtVVdY6ez0TEKjnRQ7Y6NajPskAcdTre6WY2uMH81wkKblRDaz9HlV kvQWOfF7njEGUcPNpRPTVev+dEE3qRRiFEZRtl7E306Ccp7Jzjn1keVRGR9w== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This exercises the io_uring_buf and io_uring_file iterators, and tests sparse file sets as well. Cc: Jens Axboe Cc: Pavel Begunkov Cc: io-uring@vger.kernel.org Signed-off-by: Kumar Kartikeya Dwivedi --- .../selftests/bpf/prog_tests/bpf_iter.c | 226 ++++++++++++++++++ .../selftests/bpf/progs/bpf_iter_io_uring.c | 50 ++++ 2 files changed, 276 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_io_uring.c diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_iter.c b/tools/testing/selftests/bpf/prog_tests/bpf_iter.c index 3e10abce3e5a..baf11fe2f88d 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_iter.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_iter.c @@ -1,6 +1,9 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2020 Facebook */ +#include #include +#include + #include "bpf_iter_ipv6_route.skel.h" #include "bpf_iter_netlink.skel.h" #include "bpf_iter_bpf_map.skel.h" @@ -26,6 +29,7 @@ #include "bpf_iter_bpf_sk_storage_map.skel.h" #include "bpf_iter_test_kern5.skel.h" #include "bpf_iter_test_kern6.skel.h" +#include "bpf_iter_io_uring.skel.h" static int duration; @@ -1239,6 +1243,224 @@ static void test_task_vma(void) bpf_iter_task_vma__destroy(skel); } +static int sys_io_uring_setup(u32 entries, struct io_uring_params *p) +{ + return syscall(__NR_io_uring_setup, entries, p); +} + +static int io_uring_register_bufs(int io_uring_fd, struct iovec *iovs, unsigned int nr) +{ + return syscall(__NR_io_uring_register, io_uring_fd, + IORING_REGISTER_BUFFERS, iovs, nr); +} + +static int io_uring_register_files(int io_uring_fd, int *fds, unsigned int nr) +{ + return syscall(__NR_io_uring_register, io_uring_fd, + IORING_REGISTER_FILES, fds, nr); +} + +static unsigned long long page_addr_to_pfn(unsigned long addr) +{ + int page_size = sysconf(_SC_PAGE_SIZE), fd, ret; + unsigned long long pfn; + + if (page_size < 0) + return 0; + fd = open("/proc/self/pagemap", O_RDONLY); + if (fd < 0) + return 0; + + ret = pread(fd, &pfn, sizeof(pfn), (addr / page_size) * 8); + close(fd); + if (ret < 0) + return 0; + /* Bits 0-54 have PFN for non-swapped page */ + return pfn & 0x7fffffffffffff; +} + +void test_io_uring_buf(void) +{ + DECLARE_LIBBPF_OPTS(bpf_iter_attach_opts, opts); + char rbuf[4096], buf[4096] = "B\n"; + union bpf_iter_link_info linfo; + struct bpf_iter_io_uring *skel; + int ret, fd, i, len = 128; + struct io_uring_params p; + struct iovec iovs[8]; + int iter_fd; + char *str; + + opts.link_info = &linfo; + opts.link_info_len = sizeof(linfo); + + skel = bpf_iter_io_uring__open_and_load(); + if (!ASSERT_OK_PTR(skel, "bpf_iter_io_uring__open_and_load")) + return; + + for (i = 0; i < ARRAY_SIZE(iovs); i++) { + iovs[i].iov_len = len; + iovs[i].iov_base = mmap(NULL, len, PROT_READ | PROT_WRITE, + MAP_ANONYMOUS | MAP_SHARED, -1, 0); + if (iovs[i].iov_base == MAP_FAILED) + goto end; + len *= 2; + } + + memset(&p, 0, sizeof(p)); + fd = sys_io_uring_setup(1, &p); + if (!ASSERT_GE(fd, 0, "io_uring_setup")) + goto end; + + linfo.io_uring.io_uring_fd = fd; + skel->links.dump_io_uring_buf = bpf_program__attach_iter(skel->progs.dump_io_uring_buf, + &opts); + if (!ASSERT_OK_PTR(skel->links.dump_io_uring_buf, "bpf_program__attach_iter")) + goto end_close_fd; + + ret = io_uring_register_bufs(fd, iovs, ARRAY_SIZE(iovs)); + if (!ASSERT_OK(ret, "io_uring_register_bufs")) + goto end_close_fd; + + /* "B\n" */ + len = 2; + str = buf + len; + for (int j = 0; j < ARRAY_SIZE(iovs); j++) { + ret = snprintf(str, sizeof(buf) - len, "%d:0x%lx:%zu\n", j, + (unsigned long)iovs[j].iov_base, + iovs[j].iov_len); + if (!ASSERT_GE(ret, 0, "snprintf") || !ASSERT_LT(ret, sizeof(buf) - len, "snprintf")) + goto end_close_fd; + len += ret; + str += ret; + + ret = snprintf(str, sizeof(buf) - len, "`-PFN for bvec[0]=%llu\n", + page_addr_to_pfn((unsigned long)iovs[j].iov_base)); + if (!ASSERT_GE(ret, 0, "snprintf") || !ASSERT_LT(ret, sizeof(buf) - len, "snprintf")) + goto end_close_fd; + len += ret; + str += ret; + } + + ret = snprintf(str, sizeof(buf) - len, "E:%zu\n", ARRAY_SIZE(iovs)); + if (!ASSERT_GE(ret, 0, "snprintf") || !ASSERT_LT(ret, sizeof(buf) - len, "snprintf")) + goto end_close_fd; + + iter_fd = bpf_iter_create(bpf_link__fd(skel->links.dump_io_uring_buf)); + if (!ASSERT_GE(iter_fd, 0, "bpf_iter_create")) + goto end_close_fd; + + ret = read_fd_into_buffer(iter_fd, rbuf, sizeof(rbuf)); + if (!ASSERT_GT(ret, 0, "read_fd_into_buffer")) + goto end_close_iter; + + ASSERT_OK(strcmp(rbuf, buf), "compare iterator output"); + + puts("=== Expected Output ==="); + printf("%s", buf); + puts("==== Actual Output ===="); + printf("%s", rbuf); + puts("======================="); + +end_close_iter: + close(iter_fd); +end_close_fd: + close(fd); +end: + while (i--) + munmap(iovs[i].iov_base, iovs[i].iov_len); + bpf_iter_io_uring__destroy(skel); +} + +void test_io_uring_file(void) +{ + int reg_files[] = { [0 ... 7] = -1 }; + DECLARE_LIBBPF_OPTS(bpf_iter_attach_opts, opts); + char buf[4096] = "B\n", rbuf[4096] = {}, *str; + union bpf_iter_link_info linfo = {}; + struct bpf_iter_io_uring *skel; + int iter_fd, fd, len = 0, ret; + struct io_uring_params p; + + opts.link_info = &linfo; + opts.link_info_len = sizeof(linfo); + + skel = bpf_iter_io_uring__open_and_load(); + if (!ASSERT_OK_PTR(skel, "bpf_iter_io_uring__open_and_load")) + return; + + /* "B\n" */ + len = 2; + str = buf + len; + ret = snprintf(str, sizeof(buf) - len, "B\n"); + for (int i = 0; i < ARRAY_SIZE(reg_files); i++) { + char templ[] = "/tmp/io_uringXXXXXX"; + const char *name, *def = ""; + + /* create sparse set */ + if (i & 1) { + name = def; + } else { + reg_files[i] = mkstemp(templ); + if (!ASSERT_GE(reg_files[i], 0, templ)) + goto end_close_reg_files; + name = templ; + ASSERT_OK(unlink(name), "unlink"); + } + ret = snprintf(str, sizeof(buf) - len, "%d:%s%s\n", i, name, name != def ? " (deleted)" : ""); + if (!ASSERT_GE(ret, 0, "snprintf") || !ASSERT_LT(ret, sizeof(buf) - len, "snprintf")) + goto end_close_reg_files; + len += ret; + str += ret; + } + + ret = snprintf(str, sizeof(buf) - len, "E:%zu\n", ARRAY_SIZE(reg_files)); + if (!ASSERT_GE(ret, 0, "snprintf") || !ASSERT_LT(ret, sizeof(buf) - len, "snprintf")) + goto end_close_reg_files; + + memset(&p, 0, sizeof(p)); + fd = sys_io_uring_setup(1, &p); + if (!ASSERT_GE(fd, 0, "io_uring_setup")) + goto end_close_reg_files; + + linfo.io_uring.io_uring_fd = fd; + skel->links.dump_io_uring_file = bpf_program__attach_iter(skel->progs.dump_io_uring_file, + &opts); + if (!ASSERT_OK_PTR(skel->links.dump_io_uring_file, "bpf_program__attach_iter")) + goto end_close_fd; + + iter_fd = bpf_iter_create(bpf_link__fd(skel->links.dump_io_uring_file)); + if (!ASSERT_GE(iter_fd, 0, "bpf_iter_create")) + goto end; + + ret = io_uring_register_files(fd, reg_files, ARRAY_SIZE(reg_files)); + if (!ASSERT_OK(ret, "io_uring_register_files")) + goto end_iter_fd; + + ret = read_fd_into_buffer(iter_fd, rbuf, sizeof(rbuf)); + if (!ASSERT_GT(ret, 0, "read_fd_into_buffer(iterator_fd, buf)")) + goto end_iter_fd; + + ASSERT_OK(strcmp(rbuf, buf), "compare iterator output"); + + puts("=== Expected Output ==="); + printf("%s", buf); + puts("==== Actual Output ===="); + printf("%s", rbuf); + puts("======================="); +end_iter_fd: + close(iter_fd); +end_close_fd: + close(fd); +end_close_reg_files: + for (int i = 0; i < ARRAY_SIZE(reg_files); i++) { + if (reg_files[i] != -1) + close(reg_files[i]); + } +end: + bpf_iter_io_uring__destroy(skel); +} + void test_bpf_iter(void) { if (test__start_subtest("btf_id_or_null")) @@ -1299,4 +1521,8 @@ void test_bpf_iter(void) test_rdonly_buf_out_of_bound(); if (test__start_subtest("buf-neg-offset")) test_buf_neg_offset(); + if (test__start_subtest("io_uring_buf")) + test_io_uring_buf(); + if (test__start_subtest("io_uring_file")) + test_io_uring_file(); } diff --git a/tools/testing/selftests/bpf/progs/bpf_iter_io_uring.c b/tools/testing/selftests/bpf/progs/bpf_iter_io_uring.c new file mode 100644 index 000000000000..caf8bd0bf8d4 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/bpf_iter_io_uring.c @@ -0,0 +1,50 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "bpf_iter.h" +#include + +SEC("iter/io_uring_buf") +int dump_io_uring_buf(struct bpf_iter__io_uring_buf *ctx) +{ + struct io_mapped_ubuf *ubuf = ctx->ubuf; + struct seq_file *seq = ctx->meta->seq; + unsigned int index = ctx->index; + + if (!ctx->meta->seq_num) + BPF_SEQ_PRINTF(seq, "B\n"); + + if (ubuf) { + BPF_SEQ_PRINTF(seq, "%u:0x%lx:%lu\n", index, (unsigned long)ubuf->ubuf, + (unsigned long)ubuf->ubuf_end - ubuf->ubuf); + BPF_SEQ_PRINTF(seq, "`-PFN for bvec[0]=%lu\n", + (unsigned long)bpf_page_to_pfn(ubuf->bvec[0].bv_page)); + } else { + BPF_SEQ_PRINTF(seq, "E:%u\n", index); + } + return 0; +} + +SEC("iter/io_uring_file") +int dump_io_uring_file(struct bpf_iter__io_uring_file *ctx) +{ + struct seq_file *seq = ctx->meta->seq; + unsigned int index = ctx->index; + struct file *file = ctx->file; + char buf[256] = ""; + + if (!ctx->meta->seq_num) + BPF_SEQ_PRINTF(seq, "B\n"); + /* for io_uring_file iterator, this is the terminating condition */ + if (ctx->ctx->nr_user_files == index) { + BPF_SEQ_PRINTF(seq, "E:%u\n", index); + return 0; + } + if (file) { + bpf_d_path(&file->f_path, buf, sizeof(buf)); + BPF_SEQ_PRINTF(seq, "%u:%s\n", index, buf); + } else { + BPF_SEQ_PRINTF(seq, "%u:\n", index); + } + return 0; +} + +char _license[] SEC("license") = "GPL"; From patchwork Tue Nov 16 05:42:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12621429 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69533C433F5 for ; Tue, 16 Nov 2021 05:59:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5210C61C14 for ; Tue, 16 Nov 2021 05:59:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239575AbhKPGCf (ORCPT ); Tue, 16 Nov 2021 01:02:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240226AbhKPGBn (ORCPT ); Tue, 16 Nov 2021 01:01:43 -0500 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DCB28C061198; Mon, 15 Nov 2021 21:42:57 -0800 (PST) Received: by mail-pl1-x643.google.com with SMTP id y7so16552951plp.0; Mon, 15 Nov 2021 21:42:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=e6K5BXrMyolcmdiOfyNeEh73nCV8EI1WCMaNjFKoCaQ=; b=EYMkJSXLoa0ztBWSqmFKZVu3vcoZZytjxVtqBnWFCac7xGykPEHAYM70yIZA0Hm/oD 8XT/5vnFCMyvx442ULlXiofa8GZm0jwAhEie4Ue7TD5gJFGCRncwmZm1fP68P1zUMwpw CrAjfL2chkAA0zSkQZaOcnIca21w7x1QOp5EiC6VRgzOlgzNa993212o+Y0ahsrvUh68 l2xOVy9AIxHn3CT0mbNBY21inSit6EeV7QuJhsQnFgoeyU75XKp5n3GJFVPkmEvh6p6P T7wRmzGL2drDpeMCr9k18gT9L211RFQF665orIbkic3ScASxs5Cd72+QNUGr4bGWX0nC q3kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e6K5BXrMyolcmdiOfyNeEh73nCV8EI1WCMaNjFKoCaQ=; b=47RYqeccgJ4uG9sR2mDu0leZ30x0LwaojjQi1iZCvcThdfAQfOEa1XyHzyCQMLDCgG tiXPAda3/ng4FAMn4V/Bz/S9e6QGE+44aYFHlwQl5wktYctcNaS2M2t2P7NuAi36TmoM wZrSJrk3HCH1IFCDPtv1+ipXEysTMAl8/gsBDpCWbLn9L9vkx8hQTWxx5TPczRHCHe0j WVc0H2sQxPnpngpyq9WqXcbP3lxXZw7t8NW24prhwk7gxvanUOkhoVUpoVU54gN79p2a iiPSrWbLHDDbuu+qOGBJj3taav/ejHuW+v2Km/t1RtUhXYzd4bGJElfNKKevhEgEWA+h yIJQ== X-Gm-Message-State: AOAM531WWO8C1n4EbzQUtjUfuyJswCwgS66DAgd5Ndh2A42qpr9OjfGw 7cJvJj2Jj/Ku+FWp0l3kbD2Zx0nPHDE= X-Google-Smtp-Source: ABdhPJzoT289D9hKmGhb5k4T6hxNisYvJ/8bfdr4mxR+BleXQ0qe1BChDLQP+fZFY2slZcYLnCgZUQ== X-Received: by 2002:a17:90b:38c7:: with SMTP id nn7mr5551574pjb.105.1637041377236; Mon, 15 Nov 2021 21:42:57 -0800 (PST) Received: from localhost ([2405:201:6014:d064:3d4e:6265:800c:dc84]) by smtp.gmail.com with ESMTPSA id qe12sm1087344pjb.29.2021.11.15.21.42.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 21:42:57 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexander Viro , linux-fsdevel@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Pavel Emelyanov , Alexander Mihalicyn , Andrei Vagin , criu@openvz.org, io-uring@vger.kernel.org Subject: [PATCH bpf-next v1 6/8] selftests/bpf: Add test for epoll BPF iterator Date: Tue, 16 Nov 2021 11:12:35 +0530 Message-Id: <20211116054237.100814-7-memxor@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211116054237.100814-1-memxor@gmail.com> References: <20211116054237.100814-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5777; h=from:subject; bh=DG1ceIlzLy6lY4GMZPn9QTX203OUaeTgUvIDcoyd6f8=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhk0S6d2hVzCKfLqmy8AJRA3YMmFlPTEsu6PpJAH4r n+xBa3uJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYZNEugAKCRBM4MiGSL8RypSxD/ 9Ed2jtsSmSIhILwYK0H6KRZGs4Qeu5D7rwpmcKKT1CN8CnssYKaJkIuc5dlYLZnKx3AhsNTiHUfsQe RFtRDbWWcLvYKOyZ4K6B8+vrU/XNsblWArw6YdJrwbKZjHiiRfMZAC2ZIxltlLRKVoAjdbLevvO7oH aadSywqpunQiKrNEd+s/16f+Re/NkSlgMu6nBzEyKfiFJh2rltpQxJ91M68Rj7v4VAx2njqsea//7p aeNqmaZSSNl+6QRHvDgbZ09CfV4XnEcdS3ih2dX+gEu6SjU+SzABvM5qN8TZIdinM01mKFJq+PPRzQ zTAss5Vn7xlgzSe3VZIgx78jFl1AaElj+C1eDzb69fWetH7tY/KTl4GBraXDLrWSWpWF7miXB6ys7B xowJynKQ0kUWhODV1/aeuNAYw/4B7mHBLok2QeZAnJcpNZuweW3noARX9A+JsVaF1DYVcjdogkcykH DHwe5d+1+nOp4F9y/9R/rDaMlpV2QQCqUAENmXaWlRd6O3Q80+HOu6WHPlPD7c+LkZ9HJ/S7fHCYsp /aHm4ZYNYV+Fqc3pXqIlJOkPvzAPBEUWuOJwcyHSSfSnGzVdwn1DO1RhBRkOA49JW4gZNxv/NrpRwH BPFAaforx7zO4hZm7u4WS6pWgWUjX7K8QzWEk+1mHDvdVycb8EnMEuetSvcg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This tests the epoll iterator, including peeking into the epitem to inspect the registered file and fd number, and verifying that in userspace. Cc: Alexander Viro Cc: linux-fsdevel@vger.kernel.org Signed-off-by: Kumar Kartikeya Dwivedi --- .../selftests/bpf/prog_tests/bpf_iter.c | 121 ++++++++++++++++++ .../selftests/bpf/progs/bpf_iter_epoll.c | 33 +++++ 2 files changed, 154 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_epoll.c diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_iter.c b/tools/testing/selftests/bpf/prog_tests/bpf_iter.c index baf11fe2f88d..45e186385f9a 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_iter.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_iter.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2020 Facebook */ #include +#include #include #include @@ -30,6 +31,7 @@ #include "bpf_iter_test_kern5.skel.h" #include "bpf_iter_test_kern6.skel.h" #include "bpf_iter_io_uring.skel.h" +#include "bpf_iter_epoll.skel.h" static int duration; @@ -1461,6 +1463,123 @@ void test_io_uring_file(void) bpf_iter_io_uring__destroy(skel); } +void test_epoll(void) +{ + const char *fmt = "B\npipe:%d\nsocket:%d\npipe:%d\nsocket:%d\nE\n"; + DECLARE_LIBBPF_OPTS(bpf_iter_attach_opts, opts); + char buf[4096] = {}, rbuf[4096] = {}; + union bpf_iter_link_info linfo; + int fds[2], sk[2], epfd, ret; + struct bpf_iter_epoll *skel; + struct epoll_event ev = {}; + int iter_fd, set[4]; + char *s, *t; + + opts.link_info = &linfo; + opts.link_info_len = sizeof(linfo); + + skel = bpf_iter_epoll__open_and_load(); + if (!ASSERT_OK_PTR(skel, "bpf_iter_epoll__open_and_load")) + return; + + epfd = epoll_create1(EPOLL_CLOEXEC); + if (!ASSERT_GE(epfd, 0, "epoll_create1")) + goto end; + + ret = pipe(fds); + if (!ASSERT_OK(ret, "pipe(fds)")) + goto end_epfd; + + ret = socketpair(AF_UNIX, SOCK_STREAM, 0, sk); + if (!ASSERT_OK(ret, "socketpair")) + goto end_pipe; + + ev.events = EPOLLIN; + + ret = epoll_ctl(epfd, EPOLL_CTL_ADD, fds[0], &ev); + if (!ASSERT_OK(ret, "epoll_ctl")) + goto end_sk; + + ret = epoll_ctl(epfd, EPOLL_CTL_ADD, sk[0], &ev); + if (!ASSERT_OK(ret, "epoll_ctl")) + goto end_sk; + + ret = epoll_ctl(epfd, EPOLL_CTL_ADD, fds[1], &ev); + if (!ASSERT_OK(ret, "epoll_ctl")) + goto end_sk; + + ret = epoll_ctl(epfd, EPOLL_CTL_ADD, sk[1], &ev); + if (!ASSERT_OK(ret, "epoll_ctl")) + goto end_sk; + + linfo.epoll.epoll_fd = epfd; + skel->links.dump_epoll = bpf_program__attach_iter(skel->progs.dump_epoll, &opts); + if (!ASSERT_OK_PTR(skel->links.dump_epoll, "bpf_program__attach_iter")) + goto end_sk; + + iter_fd = bpf_iter_create(bpf_link__fd(skel->links.dump_epoll)); + if (!ASSERT_GE(iter_fd, 0, "bpf_iter_create")) + goto end_sk; + + ret = epoll_ctl(epfd, EPOLL_CTL_ADD, iter_fd, &ev); + if (!ASSERT_EQ(ret, -1, "epoll_ctl add for iter_fd")) + goto end_iter_fd; + + ret = snprintf(buf, sizeof(buf), fmt, fds[0], sk[0], fds[1], sk[1]); + if (!ASSERT_GE(ret, 0, "snprintf") || !ASSERT_LT(ret, sizeof(buf), "snprintf")) + goto end_iter_fd; + + ret = read_fd_into_buffer(iter_fd, rbuf, sizeof(rbuf)); + if (!ASSERT_GT(ret, 0, "read_fd_into_buffer")) + goto end_iter_fd; + + puts("=== Expected Output ==="); + printf("%s", buf); + puts("==== Actual Output ===="); + printf("%s", rbuf); + puts("======================="); + + s = rbuf; + while ((s = strtok_r(s, "\n", &t))) { + int fd = -1; + + if (s[0] == 'B' || s[0] == 'E') + goto next; + ASSERT_EQ(sscanf(s, s[0] == 'p' ? "pipe:%d" : "socket:%d", &fd), 1, s); + if (fd == fds[0]) { + ASSERT_NEQ(set[0], 1, "pipe[0]"); + set[0] = 1; + } else if (fd == fds[1]) { + ASSERT_NEQ(set[1], 1, "pipe[1]"); + set[1] = 1; + } else if (fd == sk[0]) { + ASSERT_NEQ(set[2], 1, "sk[0]"); + set[2] = 1; + } else if (fd == sk[1]) { + ASSERT_NEQ(set[3], 1, "sk[1]"); + set[3] = 1; + } else { + ASSERT_TRUE(0, "Incorrect fd in iterator output"); + } +next: + s = NULL; + } + for (int i = 0; i < ARRAY_SIZE(set); i++) + ASSERT_EQ(set[i], 1, "fd found"); +end_iter_fd: + close(iter_fd); +end_sk: + close(sk[1]); + close(sk[0]); +end_pipe: + close(fds[1]); + close(fds[0]); +end_epfd: + close(epfd); +end: + bpf_iter_epoll__destroy(skel); +} + void test_bpf_iter(void) { if (test__start_subtest("btf_id_or_null")) @@ -1525,4 +1644,6 @@ void test_bpf_iter(void) test_io_uring_buf(); if (test__start_subtest("io_uring_file")) test_io_uring_file(); + if (test__start_subtest("epoll")) + test_epoll(); } diff --git a/tools/testing/selftests/bpf/progs/bpf_iter_epoll.c b/tools/testing/selftests/bpf/progs/bpf_iter_epoll.c new file mode 100644 index 000000000000..0afc74d154a1 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/bpf_iter_epoll.c @@ -0,0 +1,33 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "bpf_iter.h" +#include + +extern void pipefifo_fops __ksym; + +SEC("iter/epoll") +int dump_epoll(struct bpf_iter__epoll *ctx) +{ + struct seq_file *seq = ctx->meta->seq; + struct epitem *epi = ctx->epi; + char sstr[] = "socket"; + char pstr[] = "pipe"; + + if (!ctx->meta->seq_num) { + BPF_SEQ_PRINTF(seq, "B\n"); + } + if (epi) { + struct file *f = epi->ffd.file; + char *str; + + if (f->f_op == &pipefifo_fops) + str = pstr; + else + str = sstr; + BPF_SEQ_PRINTF(seq, "%s:%d\n", str, epi->ffd.fd); + } else { + BPF_SEQ_PRINTF(seq, "E\n"); + } + return 0; +} + +char _license[] SEC("license") = "GPL"; From patchwork Tue Nov 16 05:42:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12621431 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A51CC433EF for ; Tue, 16 Nov 2021 05:59:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 096FC613D0 for ; Tue, 16 Nov 2021 05:59:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237286AbhKPGCh (ORCPT ); Tue, 16 Nov 2021 01:02:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35614 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240252AbhKPGBo (ORCPT ); Tue, 16 Nov 2021 01:01:44 -0500 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF592C04A402; Mon, 15 Nov 2021 21:43:00 -0800 (PST) Received: by mail-pf1-x442.google.com with SMTP id r130so17138598pfc.1; Mon, 15 Nov 2021 21:43:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=V4/reK9eDSJ60XjxkKKa4zy03IzWT9nBfESfKcYyf04=; b=GRVqm+SZdDdHB12BlbxiItHoN6sCkL47u9sM2PYCprpn2ATAaMTBLJ5iuBlZezgoU5 mSwQ+i5H5mmndJCgMJJG/QEc9uVS/FtwAhs7kw3lfksPKJnUDxW9OlfK5H8kwFnGF9BW MxNvY6G+Up2gGgc5TGmy5jys4wU0OpzSNjrXf1VuP+AsyjPXUXI0F7m3ziEBQ0Op6M4Z 1ohZPY/omKXNh8WMYTDqz2JnPPGH6mRNAXQf5UzwlvOl4+/van31sEduV+PwHVPzlVAX L1dGluhpUypUyRZI77meksrdWH9Tm1TSwS8JtelpPwF6EhfG3NXAyt1x7d7UBAn97ivw w3lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=V4/reK9eDSJ60XjxkKKa4zy03IzWT9nBfESfKcYyf04=; b=xNHBtFbvbXXZEUCYtrr0wuGSV7Ho6HnwqUDdrQPxhW/Fyhi+Yjb74qzIMaRpxKl64k SvA7fZhhB8f2R/YES7xWlExPw0V5IyfBW9GBU4HAq2Kw+90AFFU5peZ+iwOsErnSeaSC MUvkizsiqTn2ZFhUOKyZzgJZlDSyACINUU7E8FW/f7nMO1zNdCqYLzP3rPIxQVG7iAQ3 2cwVcrB4NgyTmb8JQbGkev1bCX3A4YAvgidAIEqGPX3CX75WUpiP/fAV8NZLdwopvxdf 4WWCPVeF7OjXiMPSOJH3A4Cw3rc7o/Y1pTwMquDfNMPIoXZuhgYqXhVCp9zj4ZC0WtIY TwQw== X-Gm-Message-State: AOAM531ndu8Jwn6nDPC77d56iOYB9Men7Q2udWuwuTGXyfpF4F4VwHzW h79aF8eKyZFt5VRGkeWf1I+y28tnc70= X-Google-Smtp-Source: ABdhPJz2tgyxpi3FNkGKmAzOfdZYkYwFUDJN0bb/JI5CPJ6f8fFkp0nUeQUnkn7aYO+eYxJZpaMRQQ== X-Received: by 2002:a65:6388:: with SMTP id h8mr3163938pgv.94.1637041380137; Mon, 15 Nov 2021 21:43:00 -0800 (PST) Received: from localhost ([2405:201:6014:d064:3d4e:6265:800c:dc84]) by smtp.gmail.com with ESMTPSA id 130sm17173265pfu.170.2021.11.15.21.42.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 21:42:59 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Pavel Emelyanov , Alexander Mihalicyn , Andrei Vagin , criu@openvz.org, io-uring@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH bpf-next v1 7/8] selftests/bpf: Test partial reads for io_uring, epoll iterators Date: Tue, 16 Nov 2021 11:12:36 +0530 Message-Id: <20211116054237.100814-8-memxor@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211116054237.100814-1-memxor@gmail.com> References: <20211116054237.100814-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4142; h=from:subject; bh=mxVmYFKFc0knu3iGLQkyTonHs8thee+ZDz+UE2gs73o=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhk0S786C7TvJZRQU4B8/yCgav7Dp9RApqxGlxKiqj qzsyO5SJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYZNEuwAKCRBM4MiGSL8RyucrD/ 9uriAEGOR2B8L5vXP9r13DHstYRKHWRHjy08Fj5FfjCUTyd8tdbG9QjsNV2vkv7hSD+PO2Vf5Rx1Pf zmegncP4pD1h5oGgq8fqGGHfHtXxoY6//Vcd2AttByNLQ3xsarHR1f2YaviYZEXYNZufWOwP47JpcK EcgVITTqFPTtsPeqaXoCq4TzOJ+VIgfAEB84KvGFr36vw6UOfsMNwIgoUVYH1LDfkB07ByHuXXGdv9 oPKC5uYHP6toUmEKPjl1pWXGp7dI2MzQf/PpQSKKAueh/zPhzlzKZY5d25j29Z/PjtCpaF3gyL+8sb K9Ny25oAygXltNhiNkNCSnYjYX5RryxXqmdQ/occelTuw0pJG1lcXv4io5gkAkRySN6rVv/PU9mXxv UIMQSjRpMXD6rxk+rMT2QDbSzIgvMVkw7oiHPw2j1gUoyPJBMLV0gWBpR4stBuPwoOO8jVErZ4PskN BL0NZh7cA4ZgT44hg902y7q8FNGux8a3fJicTRkXXHdCyIHpY3VcJcclv9j6ThccMYzyibTC0o+C9H uA8mi0qXXmZBtMbqIA0lCnoeXRsi3bkoyY18GRYh4D7l4pvUU5nLOeQwaoO0en12sKbGeQF56oVKX/ X0s3F7xTZbqM11kLwJdUQe7d2d73SORMv1v1TX2WIldowebxni5K26Qpc3rQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Ensure that the output is consistent in face of partial reads that return to userspace and then resume again later. To this end, we do reads in 1-byte chunks, which is a bit stupid in real life, but works well to simulate interrupted iteration. This also tests case where seq_file buffer is consumed (after seq_printf) on interrupted read before iterator invoked BPF prog again. Signed-off-by: Kumar Kartikeya Dwivedi --- .../selftests/bpf/prog_tests/bpf_iter.c | 33 ++++++++++++------- 1 file changed, 22 insertions(+), 11 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_iter.c b/tools/testing/selftests/bpf/prog_tests/bpf_iter.c index 45e186385f9a..c27f3e10211c 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_iter.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_iter.c @@ -72,13 +72,13 @@ static void do_dummy_read(struct bpf_program *prog) bpf_link__destroy(link); } -static int read_fd_into_buffer(int fd, char *buf, int size) +static int __read_fd_into_buffer(int fd, char *buf, int size, size_t chunks) { int bufleft = size; int len; do { - len = read(fd, buf, bufleft); + len = read(fd, buf, chunks ?: bufleft); if (len > 0) { buf += len; bufleft -= len; @@ -88,6 +88,11 @@ static int read_fd_into_buffer(int fd, char *buf, int size) return len < 0 ? len : size - bufleft; } +static int read_fd_into_buffer(int fd, char *buf, int size) +{ + return __read_fd_into_buffer(fd, buf, size, 0); +} + static void test_ipv6_route(void) { struct bpf_iter_ipv6_route *skel; @@ -1281,7 +1286,7 @@ static unsigned long long page_addr_to_pfn(unsigned long addr) return pfn & 0x7fffffffffffff; } -void test_io_uring_buf(void) +void test_io_uring_buf(bool partial) { DECLARE_LIBBPF_OPTS(bpf_iter_attach_opts, opts); char rbuf[4096], buf[4096] = "B\n"; @@ -1352,7 +1357,7 @@ void test_io_uring_buf(void) if (!ASSERT_GE(iter_fd, 0, "bpf_iter_create")) goto end_close_fd; - ret = read_fd_into_buffer(iter_fd, rbuf, sizeof(rbuf)); + ret = __read_fd_into_buffer(iter_fd, rbuf, sizeof(rbuf), partial); if (!ASSERT_GT(ret, 0, "read_fd_into_buffer")) goto end_close_iter; @@ -1374,7 +1379,7 @@ void test_io_uring_buf(void) bpf_iter_io_uring__destroy(skel); } -void test_io_uring_file(void) +void test_io_uring_file(bool partial) { int reg_files[] = { [0 ... 7] = -1 }; DECLARE_LIBBPF_OPTS(bpf_iter_attach_opts, opts); @@ -1439,7 +1444,7 @@ void test_io_uring_file(void) if (!ASSERT_OK(ret, "io_uring_register_files")) goto end_iter_fd; - ret = read_fd_into_buffer(iter_fd, rbuf, sizeof(rbuf)); + ret = __read_fd_into_buffer(iter_fd, rbuf, sizeof(rbuf), partial); if (!ASSERT_GT(ret, 0, "read_fd_into_buffer(iterator_fd, buf)")) goto end_iter_fd; @@ -1463,7 +1468,7 @@ void test_io_uring_file(void) bpf_iter_io_uring__destroy(skel); } -void test_epoll(void) +void test_epoll(bool partial) { const char *fmt = "B\npipe:%d\nsocket:%d\npipe:%d\nsocket:%d\nE\n"; DECLARE_LIBBPF_OPTS(bpf_iter_attach_opts, opts); @@ -1529,7 +1534,7 @@ void test_epoll(void) if (!ASSERT_GE(ret, 0, "snprintf") || !ASSERT_LT(ret, sizeof(buf), "snprintf")) goto end_iter_fd; - ret = read_fd_into_buffer(iter_fd, rbuf, sizeof(rbuf)); + ret = __read_fd_into_buffer(iter_fd, rbuf, sizeof(rbuf), partial); if (!ASSERT_GT(ret, 0, "read_fd_into_buffer")) goto end_iter_fd; @@ -1641,9 +1646,15 @@ void test_bpf_iter(void) if (test__start_subtest("buf-neg-offset")) test_buf_neg_offset(); if (test__start_subtest("io_uring_buf")) - test_io_uring_buf(); + test_io_uring_buf(false); if (test__start_subtest("io_uring_file")) - test_io_uring_file(); + test_io_uring_file(false); if (test__start_subtest("epoll")) - test_epoll(); + test_epoll(false); + if (test__start_subtest("io_uring_buf-partial")) + test_io_uring_buf(true); + if (test__start_subtest("io_uring_file-partial")) + test_io_uring_file(true); + if (test__start_subtest("epoll-partial")) + test_epoll(true); } From patchwork Tue Nov 16 05:42:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12621433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1148AC433FE for ; Tue, 16 Nov 2021 05:59:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F3AFF613D0 for ; Tue, 16 Nov 2021 05:59:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240250AbhKPGCi (ORCPT ); Tue, 16 Nov 2021 01:02:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240253AbhKPGBo (ORCPT ); Tue, 16 Nov 2021 01:01:44 -0500 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B8D1C04A403; Mon, 15 Nov 2021 21:43:03 -0800 (PST) Received: by mail-pj1-x1044.google.com with SMTP id h24so14828435pjq.2; Mon, 15 Nov 2021 21:43:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xNMq9GJJA7CIRiH95ibik9uHs9CfsmAe2wbOXiKepoU=; b=DzKnaQB+wUFswdwz06sOh6UuNIMNlcZRdYQsNZBecS2AIpjnisA6sds54G2OSNl7OW 0BXKzD7OAl+lmAWARnW6ZtymVwexFT9BaqU4PAfy9t5rcW8HUA97MKvP3G23I5YkxW/A zD2s8N/Psyf/mj0nQNl7HbL6lCk/vQTkna3R5uecrU0AYlxilcZPsUdmisttgRuHzLE/ qwTdobzPkDJDmJSbxk/lMw568IqQnD2NfCjdWeo0HwLwSo+jbqDCiSXrOpT0pqpV0n39 z9L1wob7VRPvtHXqJsPegehHTGfEM12hgdYubcl2FgG7qKY5KPV3+UpBH0bGrzsyGuRN z77Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xNMq9GJJA7CIRiH95ibik9uHs9CfsmAe2wbOXiKepoU=; b=N8VNYrur8Jew/JG2Ue5FKfcFFFQDJBMa+tFgt6tgoQPkl4saJBlsul8YcGptm1dhY5 EQqIFqDNoJXW1pJH8hN1famPMAgWuOKf86Cv8VBcbBpl9p+fE98cx2EIgSjmi4KSgVKo OPeqroHl/HbyzbTSayJC8tPZ0NOk49GzldD7FCPDcGNBRHyouABUshaxheTDGUZFQEM1 /EMaRD95z9QJONmLWE8dJL9z5HIL/tOU3K369q+0NdAGuHzfM/25KJq5ZMtLLIKviPNE d7sorCnWmGeQSa0GXHEOSN1UhZoFPDchEuJIVQCuCqmSUdpWazB1VCtD3+KfrMjO3H16 P9rA== X-Gm-Message-State: AOAM5334azeaUAkZHP3/ThHEcdWN/eJJJr50lsi2dlo5yWIB7KFHl4sm O409M22mtwVuEr8OW9vIMZ/wM6Z4gk0= X-Google-Smtp-Source: ABdhPJy4/Tbd2aBvDPZSWjue0LOPAsIvPxsbhzWx8MaZWaM8Q0AGneg204Vq85uE5NeNSMe0eAfTCw== X-Received: by 2002:a17:90a:c091:: with SMTP id o17mr5526836pjs.35.1637041383082; Mon, 15 Nov 2021 21:43:03 -0800 (PST) Received: from localhost ([2405:201:6014:d064:3d4e:6265:800c:dc84]) by smtp.gmail.com with ESMTPSA id mm22sm1015953pjb.28.2021.11.15.21.43.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 21:43:02 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Pavel Emelyanov , Alexander Mihalicyn , Andrei Vagin , criu@openvz.org, io-uring@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH bpf-next v1 8/8] selftests/bpf: Fix btf_dump test for bpf_iter_link_info Date: Tue, 16 Nov 2021 11:12:37 +0530 Message-Id: <20211116054237.100814-9-memxor@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211116054237.100814-1-memxor@gmail.com> References: <20211116054237.100814-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1199; h=from:subject; bh=2OeAqoVgm3UE8yo/6xItJyebkuS6bvyJvQVD3m8v6N4=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhk0S7cp9vrG4vOU0yZvr5SpV+sCHwxBvUfioYCwMq 0lXCH8qJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYZNEuwAKCRBM4MiGSL8RyksmD/ 9ykap1OxzGchcNWyb6qBJwWVb2/xIlSSGHWzo99rRvEZvoNlQdwX5ys7GV/8m7U6NLzhx0w1MrQG4O +WvaLbLSff4Gb2Q8DfmQug+XAF26EFhhd6E3VI2h8qR4CJ5uhbqvZp9q+o/lYlb2tUWY32dXcZnvYq bQXA/3FkSofOhUKPq5/CHTic8w69GAs59wugR3GlmQVTDT38STrcptrKCPXQ5OVpWKP8cU8HqGPgn+ fAw0KzQ+sdnzaNMW5eA+eD3WUC3wbKkLO78kcYFaA9/QtHq5Ulb6YtO2t/71Uh7F2qhdLX9o+RhsFz 3h0yPPR+ihC46gpovjzl2tITdUr/TK9ORgGTf0YwRrukGtqiNKk2VUuBABngKOTIZ0aOeV4MLLlfrH bWmpzZ1NKGdwra3g+fzNOKY25D77sQw+Ckhoh9XenHs/2XREDdHeNYgqmDnsGjFW8gmqm0PRG/kRNO A8/MezsfRwxNo8vwDY1XXqdm0nABIntz/w+IRpyvPxYff0jAXUKz+V+6E51EmrRJ+BdaM/FLDd0Op4 saQP5nKttNeElk2g8pjLpPKgczDO7qYB+tPZs0gohBu1PRkPHwLCFDlWFIbWuCuB2ym+jo9Do4Hwe1 WkhAUwcTS0HSDOOXxgMaTS7dnnNy05Yoy373veKRNJNhyTpJF/LjfBEuO5fA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Since we changed the definition while adding io_uring and epoll iterator support, adjust the selftest to check against the updated definition. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/testing/selftests/bpf/prog_tests/btf_dump.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/prog_tests/btf_dump.c b/tools/testing/selftests/bpf/prog_tests/btf_dump.c index d6272013a5a3..a2fc006e074a 100644 --- a/tools/testing/selftests/bpf/prog_tests/btf_dump.c +++ b/tools/testing/selftests/bpf/prog_tests/btf_dump.c @@ -736,7 +736,9 @@ static void test_btf_dump_struct_data(struct btf *btf, struct btf_dump *d, /* union with nested struct */ TEST_BTF_DUMP_DATA(btf, d, "union", str, union bpf_iter_link_info, BTF_F_COMPACT, - "(union bpf_iter_link_info){.map = (struct){.map_fd = (__u32)1,},}", + "(union bpf_iter_link_info){.map = (struct){.map_fd = (__u32)1,}," + ".io_uring = (struct){.io_uring_fd = (__u32)1,}," + ".epoll = (struct){.epoll_fd = (__u32)1,},}", { .map = { .map_fd = 1 }}); /* struct skb with nested structs/unions; because type output is so