diff mbox series

[bpf-next,v4,2/6] netfilter: bpf: Support BPF_F_NETFILTER_IP_DEFRAG in netfilter link

Message ID d3b0ff95c58356192ea3b50824f8cdbf02c354e3.1689203090.git.dxu@dxuuu.xyz (mailing list archive)
State Superseded
Delegated to: BPF
Headers show
Series Support defragmenting IPv(4|6) packets in BPF | expand

Checks

Context Check Description
bpf/vmtest-bpf-next-PR success PR summary
bpf/vmtest-bpf-next-VM_Test-8 success Logs for set-matrix
bpf/vmtest-bpf-next-VM_Test-30 success Logs for test_verifier on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-31 success Logs for veristat
bpf/vmtest-bpf-next-VM_Test-1 success Logs for ShellCheck
bpf/vmtest-bpf-next-VM_Test-6 success Logs for set-matrix
bpf/vmtest-bpf-next-VM_Test-2 success Logs for build for aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-4 success Logs for build for x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-5 success Logs for build for x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-3 success Logs for build for s390x with gcc
bpf/vmtest-bpf-next-VM_Test-25 success Logs for test_verifier on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-27 success Logs for test_verifier on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-28 success Logs for test_verifier on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-29 success Logs for veristat
bpf/vmtest-bpf-next-VM_Test-7 success Logs for test_maps on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-9 success Logs for test_maps on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-10 success Logs for test_maps on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-11 success Logs for test_progs on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-13 success Logs for test_progs on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-14 success Logs for test_progs on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-15 success Logs for test_progs_no_alu32 on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-17 success Logs for test_progs_no_alu32 on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-18 success Logs for test_progs_no_alu32 on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-19 success Logs for test_progs_no_alu32_parallel on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-20 success Logs for test_progs_no_alu32_parallel on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-21 success Logs for test_progs_no_alu32_parallel on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-22 success Logs for test_progs_parallel on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-23 success Logs for test_progs_parallel on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-24 success Logs for test_progs_parallel on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-26 success Logs for test_verifier on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-16 success Logs for test_progs_no_alu32 on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-12 success Logs for test_progs on s390x with gcc
netdev/series_format success Posting correctly formatted
netdev/tree_selection success Clearly marked for bpf-next, async
netdev/fixes_present success Fixes tag not required for -next series
netdev/header_inline success No static functions without inline keyword in header files
netdev/build_32bit success Errors and warnings before: 1736 this patch: 1736
netdev/cc_maintainers success CCed 22 of 22 maintainers
netdev/build_clang fail Errors and warnings before: 186 this patch: 183
netdev/verify_signedoff success Signed-off-by tag matches author and committer
netdev/deprecated_api success None detected
netdev/check_selftest success No net selftest shell script
netdev/verify_fixes success No Fixes tag
netdev/build_allmodconfig_warn success Errors and warnings before: 1735 this patch: 1735
netdev/checkpatch warning CHECK: Prefer using the BIT macro WARNING: Duplicate signature WARNING: line length of 83 exceeds 80 columns WARNING: line length of 86 exceeds 80 columns
netdev/kdoc success Errors and warnings before: 0 this patch: 0
netdev/source_inline success Was 0 now: 0

Commit Message

Daniel Xu July 12, 2023, 11:43 p.m. UTC 
This commit adds support for enabling IP defrag using pre-existing
netfilter defrag support. Basically all the flag does is bump a refcnt
while the link the active. Checks are also added to ensure the prog
requesting defrag support is run _after_ netfilter defrag hooks.

Reviewed-by: Florian Westphal <fw@strlen.de>
Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
---
 include/uapi/linux/bpf.h       |   5 ++
 net/netfilter/nf_bpf_link.c    | 129 ++++++++++++++++++++++++++++++---
 tools/include/uapi/linux/bpf.h |   5 ++
 3 files changed, 128 insertions(+), 11 deletions(-)

Comments

Alexei Starovoitov July 13, 2023, 12:43 a.m. UTC | #1 
On Wed, Jul 12, 2023 at 4:44 PM Daniel Xu <dxu@dxuuu.xyz> wrote:
> +#if IS_ENABLED(CONFIG_NF_DEFRAG_IPV6)
> +       case NFPROTO_IPV6:
> +               rcu_read_lock();
> +               v6_hook = rcu_dereference(nf_defrag_v6_hook);
> +               if (!v6_hook) {
> +                       rcu_read_unlock();
> +                       err = request_module("nf_defrag_ipv6");
> +                       if (err)
> +                               return err < 0 ? err : -EINVAL;
> +
> +                       rcu_read_lock();
> +                       v6_hook = rcu_dereference(nf_defrag_v6_hook);
> +                       if (!v6_hook) {
> +                               WARN_ONCE(1, "nf_defrag_ipv6_hooks bad registration");
> +                               err = -ENOENT;
> +                               goto out_v6;
> +                       }
> +               }
> +
> +               err = v6_hook->enable(link->net);

I was about to apply, but luckily caught this issue in my local test:

[   18.462448] BUG: sleeping function called from invalid context at
kernel/locking/mutex.c:283
[   18.463238] in_atomic(): 0, irqs_disabled(): 0, non_block: 0, pid:
2042, name: test_progs
[   18.463927] preempt_count: 0, expected: 0
[   18.464249] RCU nest depth: 1, expected: 0
[   18.464631] CPU: 15 PID: 2042 Comm: test_progs Tainted: G
O       6.4.0-04319-g6f6ec4fa00dc #4896
[   18.465480] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS rel-1.12.0-59-gc9ba5276e321-prebuilt.qemu.org 04/01/2014
[   18.466531] Call Trace:
[   18.466767]  <TASK>
[   18.466975]  dump_stack_lvl+0x32/0x40
[   18.467325]  __might_resched+0x129/0x180
[   18.467691]  mutex_lock+0x1a/0x40
[   18.468057]  nf_defrag_ipv4_enable+0x16/0x70
[   18.468467]  bpf_nf_link_attach+0x141/0x300
[   18.468856]  __sys_bpf+0x133e/0x26d0

You cannot call mutex under rcu_read_lock.

Please make sure you have all kernel debug flags on in your testing.
Daniel Xu July 13, 2023, 1:22 a.m. UTC | #2 
Hi Alexei,

On Wed, Jul 12, 2023 at 05:43:49PM -0700, Alexei Starovoitov wrote:
> On Wed, Jul 12, 2023 at 4:44 PM Daniel Xu <dxu@dxuuu.xyz> wrote:
> > +#if IS_ENABLED(CONFIG_NF_DEFRAG_IPV6)
> > +       case NFPROTO_IPV6:
> > +               rcu_read_lock();
> > +               v6_hook = rcu_dereference(nf_defrag_v6_hook);
> > +               if (!v6_hook) {
> > +                       rcu_read_unlock();
> > +                       err = request_module("nf_defrag_ipv6");
> > +                       if (err)
> > +                               return err < 0 ? err : -EINVAL;
> > +
> > +                       rcu_read_lock();
> > +                       v6_hook = rcu_dereference(nf_defrag_v6_hook);
> > +                       if (!v6_hook) {
> > +                               WARN_ONCE(1, "nf_defrag_ipv6_hooks bad registration");
> > +                               err = -ENOENT;
> > +                               goto out_v6;
> > +                       }
> > +               }
> > +
> > +               err = v6_hook->enable(link->net);
> 
> I was about to apply, but luckily caught this issue in my local test:
> 
> [   18.462448] BUG: sleeping function called from invalid context at
> kernel/locking/mutex.c:283
> [   18.463238] in_atomic(): 0, irqs_disabled(): 0, non_block: 0, pid:
> 2042, name: test_progs
> [   18.463927] preempt_count: 0, expected: 0
> [   18.464249] RCU nest depth: 1, expected: 0
> [   18.464631] CPU: 15 PID: 2042 Comm: test_progs Tainted: G
> O       6.4.0-04319-g6f6ec4fa00dc #4896
> [   18.465480] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> BIOS rel-1.12.0-59-gc9ba5276e321-prebuilt.qemu.org 04/01/2014
> [   18.466531] Call Trace:
> [   18.466767]  <TASK>
> [   18.466975]  dump_stack_lvl+0x32/0x40
> [   18.467325]  __might_resched+0x129/0x180
> [   18.467691]  mutex_lock+0x1a/0x40
> [   18.468057]  nf_defrag_ipv4_enable+0x16/0x70
> [   18.468467]  bpf_nf_link_attach+0x141/0x300
> [   18.468856]  __sys_bpf+0x133e/0x26d0
> 
> You cannot call mutex under rcu_read_lock.

Whoops, my bad. I think this patch should fix it:

```
>From 7e8927c44452db07ddd7cf0e30bb49215fc044ed Mon Sep 17 00:00:00 2001
Message-ID: <7e8927c44452db07ddd7cf0e30bb49215fc044ed.1689211250.git.dxu@dxuuu.xyz>
From: Daniel Xu <dxu@dxuuu.xyz>
Date: Wed, 12 Jul 2023 19:17:35 -0600
Subject: [PATCH] netfilter: bpf: Don't hold rcu_read_lock during
 enable/disable

->enable()/->disable() takes a mutex which can sleep. You can't sleep
during RCU read side critical section.

Our refcnt on the module will protect us from ->enable()/->disable()
from going away while we call it.

Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
---
 net/netfilter/nf_bpf_link.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/net/netfilter/nf_bpf_link.c b/net/netfilter/nf_bpf_link.c
index 77ffbf26ba3d..79704cc596aa 100644
--- a/net/netfilter/nf_bpf_link.c
+++ b/net/netfilter/nf_bpf_link.c
@@ -60,9 +60,12 @@ static int bpf_nf_enable_defrag(struct bpf_nf_link *link)
                        goto out_v4;
                }

+               rcu_read_unlock();
                err = v4_hook->enable(link->net);
                if (err)
                        module_put(v4_hook->owner);
+
+               return err;
 out_v4:
                rcu_read_unlock();
                return err;
@@ -92,9 +95,12 @@ static int bpf_nf_enable_defrag(struct bpf_nf_link *link)
                        goto out_v6;
                }

+               rcu_read_unlock();
                err = v6_hook->enable(link->net);
                if (err)
                        module_put(v6_hook->owner);
+
+               return err;
 out_v6:
                rcu_read_unlock();
                return err;
@@ -114,11 +120,11 @@ static void bpf_nf_disable_defrag(struct bpf_nf_link *link)
        case NFPROTO_IPV4:
                rcu_read_lock();
                v4_hook = rcu_dereference(nf_defrag_v4_hook);
+               rcu_read_unlock();
                if (v4_hook) {
                        v4_hook->disable(link->net);
                        module_put(v4_hook->owner);
                }
-               rcu_read_unlock();

                break;
 #endif
@@ -126,11 +132,11 @@ static void bpf_nf_disable_defrag(struct bpf_nf_link *link)
        case NFPROTO_IPV6:
                rcu_read_lock();
                v6_hook = rcu_dereference(nf_defrag_v6_hook);
+               rcu_read_unlock();
                if (v6_hook) {
                        v6_hook->disable(link->net);
                        module_put(v6_hook->owner);
                }
-               rcu_read_unlock();

                break;
        }
--
2.41.0
```

I'll send out a v5 tomorrow morning unless you feel like applying the
series + this patch today.

> 
> Please make sure you have all kernel debug flags on in your testing.
> 

Ack. Will make sure lockdep is on.


Thanks,
Daniel
Alexei Starovoitov July 13, 2023, 1:26 a.m. UTC | #3 
On Wed, Jul 12, 2023 at 6:22 PM Daniel Xu <dxu@dxuuu.xyz> wrote:
>
> Hi Alexei,
>
> On Wed, Jul 12, 2023 at 05:43:49PM -0700, Alexei Starovoitov wrote:
> > On Wed, Jul 12, 2023 at 4:44 PM Daniel Xu <dxu@dxuuu.xyz> wrote:
> > > +#if IS_ENABLED(CONFIG_NF_DEFRAG_IPV6)
> > > +       case NFPROTO_IPV6:
> > > +               rcu_read_lock();
> > > +               v6_hook = rcu_dereference(nf_defrag_v6_hook);
> > > +               if (!v6_hook) {
> > > +                       rcu_read_unlock();
> > > +                       err = request_module("nf_defrag_ipv6");
> > > +                       if (err)
> > > +                               return err < 0 ? err : -EINVAL;
> > > +
> > > +                       rcu_read_lock();
> > > +                       v6_hook = rcu_dereference(nf_defrag_v6_hook);
> > > +                       if (!v6_hook) {
> > > +                               WARN_ONCE(1, "nf_defrag_ipv6_hooks bad registration");
> > > +                               err = -ENOENT;
> > > +                               goto out_v6;
> > > +                       }
> > > +               }
> > > +
> > > +               err = v6_hook->enable(link->net);
> >
> > I was about to apply, but luckily caught this issue in my local test:
> >
> > [   18.462448] BUG: sleeping function called from invalid context at
> > kernel/locking/mutex.c:283
> > [   18.463238] in_atomic(): 0, irqs_disabled(): 0, non_block: 0, pid:
> > 2042, name: test_progs
> > [   18.463927] preempt_count: 0, expected: 0
> > [   18.464249] RCU nest depth: 1, expected: 0
> > [   18.464631] CPU: 15 PID: 2042 Comm: test_progs Tainted: G
> > O       6.4.0-04319-g6f6ec4fa00dc #4896
> > [   18.465480] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> > BIOS rel-1.12.0-59-gc9ba5276e321-prebuilt.qemu.org 04/01/2014
> > [   18.466531] Call Trace:
> > [   18.466767]  <TASK>
> > [   18.466975]  dump_stack_lvl+0x32/0x40
> > [   18.467325]  __might_resched+0x129/0x180
> > [   18.467691]  mutex_lock+0x1a/0x40
> > [   18.468057]  nf_defrag_ipv4_enable+0x16/0x70
> > [   18.468467]  bpf_nf_link_attach+0x141/0x300
> > [   18.468856]  __sys_bpf+0x133e/0x26d0
> >
> > You cannot call mutex under rcu_read_lock.
>
> Whoops, my bad. I think this patch should fix it:
>
> ```
> From 7e8927c44452db07ddd7cf0e30bb49215fc044ed Mon Sep 17 00:00:00 2001
> Message-ID: <7e8927c44452db07ddd7cf0e30bb49215fc044ed.1689211250.git.dxu@dxuuu.xyz>
> From: Daniel Xu <dxu@dxuuu.xyz>
> Date: Wed, 12 Jul 2023 19:17:35 -0600
> Subject: [PATCH] netfilter: bpf: Don't hold rcu_read_lock during
>  enable/disable
>
> ->enable()/->disable() takes a mutex which can sleep. You can't sleep
> during RCU read side critical section.
>
> Our refcnt on the module will protect us from ->enable()/->disable()
> from going away while we call it.
>
> Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
> ---
>  net/netfilter/nf_bpf_link.c | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/net/netfilter/nf_bpf_link.c b/net/netfilter/nf_bpf_link.c
> index 77ffbf26ba3d..79704cc596aa 100644
> --- a/net/netfilter/nf_bpf_link.c
> +++ b/net/netfilter/nf_bpf_link.c
> @@ -60,9 +60,12 @@ static int bpf_nf_enable_defrag(struct bpf_nf_link *link)
>                         goto out_v4;
>                 }
>
> +               rcu_read_unlock();
>                 err = v4_hook->enable(link->net);
>                 if (err)
>                         module_put(v4_hook->owner);
> +
> +               return err;
>  out_v4:
>                 rcu_read_unlock();
>                 return err;
> @@ -92,9 +95,12 @@ static int bpf_nf_enable_defrag(struct bpf_nf_link *link)
>                         goto out_v6;
>                 }
>
> +               rcu_read_unlock();
>                 err = v6_hook->enable(link->net);
>                 if (err)
>                         module_put(v6_hook->owner);
> +
> +               return err;
>  out_v6:
>                 rcu_read_unlock();
>                 return err;
> @@ -114,11 +120,11 @@ static void bpf_nf_disable_defrag(struct bpf_nf_link *link)
>         case NFPROTO_IPV4:
>                 rcu_read_lock();
>                 v4_hook = rcu_dereference(nf_defrag_v4_hook);
> +               rcu_read_unlock();
>                 if (v4_hook) {
>                         v4_hook->disable(link->net);
>                         module_put(v4_hook->owner);
>                 }
> -               rcu_read_unlock();
>
>                 break;
>  #endif
> @@ -126,11 +132,11 @@ static void bpf_nf_disable_defrag(struct bpf_nf_link *link)
>         case NFPROTO_IPV6:
>                 rcu_read_lock();
>                 v6_hook = rcu_dereference(nf_defrag_v6_hook);
> +               rcu_read_unlock();

No. v6_hook is gone as soon as you unlock it.
Daniel Xu July 13, 2023, 4:33 a.m. UTC | #4 
On Wed, Jul 12, 2023 at 06:26:13PM -0700, Alexei Starovoitov wrote:
> On Wed, Jul 12, 2023 at 6:22 PM Daniel Xu <dxu@dxuuu.xyz> wrote:
> >
> > Hi Alexei,
> >
> > On Wed, Jul 12, 2023 at 05:43:49PM -0700, Alexei Starovoitov wrote:
> > > On Wed, Jul 12, 2023 at 4:44 PM Daniel Xu <dxu@dxuuu.xyz> wrote:
> > > > +#if IS_ENABLED(CONFIG_NF_DEFRAG_IPV6)
> > > > +       case NFPROTO_IPV6:
> > > > +               rcu_read_lock();
> > > > +               v6_hook = rcu_dereference(nf_defrag_v6_hook);
> > > > +               if (!v6_hook) {
> > > > +                       rcu_read_unlock();
> > > > +                       err = request_module("nf_defrag_ipv6");
> > > > +                       if (err)
> > > > +                               return err < 0 ? err : -EINVAL;
> > > > +
> > > > +                       rcu_read_lock();
> > > > +                       v6_hook = rcu_dereference(nf_defrag_v6_hook);
> > > > +                       if (!v6_hook) {
> > > > +                               WARN_ONCE(1, "nf_defrag_ipv6_hooks bad registration");
> > > > +                               err = -ENOENT;
> > > > +                               goto out_v6;
> > > > +                       }
> > > > +               }
> > > > +
> > > > +               err = v6_hook->enable(link->net);
> > >
> > > I was about to apply, but luckily caught this issue in my local test:
> > >
> > > [   18.462448] BUG: sleeping function called from invalid context at
> > > kernel/locking/mutex.c:283
> > > [   18.463238] in_atomic(): 0, irqs_disabled(): 0, non_block: 0, pid:
> > > 2042, name: test_progs
> > > [   18.463927] preempt_count: 0, expected: 0
> > > [   18.464249] RCU nest depth: 1, expected: 0
> > > [   18.464631] CPU: 15 PID: 2042 Comm: test_progs Tainted: G
> > > O       6.4.0-04319-g6f6ec4fa00dc #4896
> > > [   18.465480] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> > > BIOS rel-1.12.0-59-gc9ba5276e321-prebuilt.qemu.org 04/01/2014
> > > [   18.466531] Call Trace:
> > > [   18.466767]  <TASK>
> > > [   18.466975]  dump_stack_lvl+0x32/0x40
> > > [   18.467325]  __might_resched+0x129/0x180
> > > [   18.467691]  mutex_lock+0x1a/0x40
> > > [   18.468057]  nf_defrag_ipv4_enable+0x16/0x70
> > > [   18.468467]  bpf_nf_link_attach+0x141/0x300
> > > [   18.468856]  __sys_bpf+0x133e/0x26d0
> > >
> > > You cannot call mutex under rcu_read_lock.
> >
> > Whoops, my bad. I think this patch should fix it:
> >
> > ```
> > From 7e8927c44452db07ddd7cf0e30bb49215fc044ed Mon Sep 17 00:00:00 2001
> > Message-ID: <7e8927c44452db07ddd7cf0e30bb49215fc044ed.1689211250.git.dxu@dxuuu.xyz>
> > From: Daniel Xu <dxu@dxuuu.xyz>
> > Date: Wed, 12 Jul 2023 19:17:35 -0600
> > Subject: [PATCH] netfilter: bpf: Don't hold rcu_read_lock during
> >  enable/disable
> >
> > ->enable()/->disable() takes a mutex which can sleep. You can't sleep
> > during RCU read side critical section.
> >
> > Our refcnt on the module will protect us from ->enable()/->disable()
> > from going away while we call it.
> >
> > Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
> > ---
> >  net/netfilter/nf_bpf_link.c | 10 ++++++++--
> >  1 file changed, 8 insertions(+), 2 deletions(-)
> >
> > diff --git a/net/netfilter/nf_bpf_link.c b/net/netfilter/nf_bpf_link.c
> > index 77ffbf26ba3d..79704cc596aa 100644
> > --- a/net/netfilter/nf_bpf_link.c
> > +++ b/net/netfilter/nf_bpf_link.c
> > @@ -60,9 +60,12 @@ static int bpf_nf_enable_defrag(struct bpf_nf_link *link)
> >                         goto out_v4;
> >                 }
> >
> > +               rcu_read_unlock();
> >                 err = v4_hook->enable(link->net);
> >                 if (err)
> >                         module_put(v4_hook->owner);
> > +
> > +               return err;
> >  out_v4:
> >                 rcu_read_unlock();
> >                 return err;
> > @@ -92,9 +95,12 @@ static int bpf_nf_enable_defrag(struct bpf_nf_link *link)
> >                         goto out_v6;
> >                 }
> >
> > +               rcu_read_unlock();
> >                 err = v6_hook->enable(link->net);
> >                 if (err)
> >                         module_put(v6_hook->owner);
> > +
> > +               return err;
> >  out_v6:
> >                 rcu_read_unlock();
> >                 return err;
> > @@ -114,11 +120,11 @@ static void bpf_nf_disable_defrag(struct bpf_nf_link *link)
> >         case NFPROTO_IPV4:
> >                 rcu_read_lock();
> >                 v4_hook = rcu_dereference(nf_defrag_v4_hook);
> > +               rcu_read_unlock();
> >                 if (v4_hook) {
> >                         v4_hook->disable(link->net);
> >                         module_put(v4_hook->owner);
> >                 }
> > -               rcu_read_unlock();
> >
> >                 break;
> >  #endif
> > @@ -126,11 +132,11 @@ static void bpf_nf_disable_defrag(struct bpf_nf_link *link)
> >         case NFPROTO_IPV6:
> >                 rcu_read_lock();
> >                 v6_hook = rcu_dereference(nf_defrag_v6_hook);
> > +               rcu_read_unlock();
> 
> No. v6_hook is gone as soon as you unlock it.

I think we're protected here by the try_module_get() on the enable path.
And we only disable defrag if enabling succeeds. The module shouldn't
be able to deregister its hooks until we call the module_put() later.

I think READ_ONCE() would've been more appropriate but I wasn't sure if
that was ok given nf_defrag_v(4|6)_hook is written to by
rcu_assign_pointer() and I was assuming symmetry is necessary.

Does that sound right?

Thanks,
Daniel
Alexei Starovoitov July 13, 2023, 11:10 p.m. UTC | #5 
On Wed, Jul 12, 2023 at 9:33 PM Daniel Xu <dxu@dxuuu.xyz> wrote:
>
> On Wed, Jul 12, 2023 at 06:26:13PM -0700, Alexei Starovoitov wrote:
> > On Wed, Jul 12, 2023 at 6:22 PM Daniel Xu <dxu@dxuuu.xyz> wrote:
> > >
> > > Hi Alexei,
> > >
> > > On Wed, Jul 12, 2023 at 05:43:49PM -0700, Alexei Starovoitov wrote:
> > > > On Wed, Jul 12, 2023 at 4:44 PM Daniel Xu <dxu@dxuuu.xyz> wrote:
> > > > > +#if IS_ENABLED(CONFIG_NF_DEFRAG_IPV6)
> > > > > +       case NFPROTO_IPV6:
> > > > > +               rcu_read_lock();
> > > > > +               v6_hook = rcu_dereference(nf_defrag_v6_hook);
> > > > > +               if (!v6_hook) {
> > > > > +                       rcu_read_unlock();
> > > > > +                       err = request_module("nf_defrag_ipv6");
> > > > > +                       if (err)
> > > > > +                               return err < 0 ? err : -EINVAL;
> > > > > +
> > > > > +                       rcu_read_lock();
> > > > > +                       v6_hook = rcu_dereference(nf_defrag_v6_hook);
> > > > > +                       if (!v6_hook) {
> > > > > +                               WARN_ONCE(1, "nf_defrag_ipv6_hooks bad registration");
> > > > > +                               err = -ENOENT;
> > > > > +                               goto out_v6;
> > > > > +                       }
> > > > > +               }
> > > > > +
> > > > > +               err = v6_hook->enable(link->net);
> > > >
> > > > I was about to apply, but luckily caught this issue in my local test:
> > > >
> > > > [   18.462448] BUG: sleeping function called from invalid context at
> > > > kernel/locking/mutex.c:283
> > > > [   18.463238] in_atomic(): 0, irqs_disabled(): 0, non_block: 0, pid:
> > > > 2042, name: test_progs
> > > > [   18.463927] preempt_count: 0, expected: 0
> > > > [   18.464249] RCU nest depth: 1, expected: 0
> > > > [   18.464631] CPU: 15 PID: 2042 Comm: test_progs Tainted: G
> > > > O       6.4.0-04319-g6f6ec4fa00dc #4896
> > > > [   18.465480] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> > > > BIOS rel-1.12.0-59-gc9ba5276e321-prebuilt.qemu.org 04/01/2014
> > > > [   18.466531] Call Trace:
> > > > [   18.466767]  <TASK>
> > > > [   18.466975]  dump_stack_lvl+0x32/0x40
> > > > [   18.467325]  __might_resched+0x129/0x180
> > > > [   18.467691]  mutex_lock+0x1a/0x40
> > > > [   18.468057]  nf_defrag_ipv4_enable+0x16/0x70
> > > > [   18.468467]  bpf_nf_link_attach+0x141/0x300
> > > > [   18.468856]  __sys_bpf+0x133e/0x26d0
> > > >
> > > > You cannot call mutex under rcu_read_lock.
> > >
> > > Whoops, my bad. I think this patch should fix it:
> > >
> > > ```
> > > From 7e8927c44452db07ddd7cf0e30bb49215fc044ed Mon Sep 17 00:00:00 2001
> > > Message-ID: <7e8927c44452db07ddd7cf0e30bb49215fc044ed.1689211250.git.dxu@dxuuu.xyz>
> > > From: Daniel Xu <dxu@dxuuu.xyz>
> > > Date: Wed, 12 Jul 2023 19:17:35 -0600
> > > Subject: [PATCH] netfilter: bpf: Don't hold rcu_read_lock during
> > >  enable/disable
> > >
> > > ->enable()/->disable() takes a mutex which can sleep. You can't sleep
> > > during RCU read side critical section.
> > >
> > > Our refcnt on the module will protect us from ->enable()/->disable()
> > > from going away while we call it.
> > >
> > > Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
> > > ---
> > >  net/netfilter/nf_bpf_link.c | 10 ++++++++--
> > >  1 file changed, 8 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/net/netfilter/nf_bpf_link.c b/net/netfilter/nf_bpf_link.c
> > > index 77ffbf26ba3d..79704cc596aa 100644
> > > --- a/net/netfilter/nf_bpf_link.c
> > > +++ b/net/netfilter/nf_bpf_link.c
> > > @@ -60,9 +60,12 @@ static int bpf_nf_enable_defrag(struct bpf_nf_link *link)
> > >                         goto out_v4;
> > >                 }
> > >
> > > +               rcu_read_unlock();
> > >                 err = v4_hook->enable(link->net);
> > >                 if (err)
> > >                         module_put(v4_hook->owner);
> > > +
> > > +               return err;
> > >  out_v4:
> > >                 rcu_read_unlock();
> > >                 return err;
> > > @@ -92,9 +95,12 @@ static int bpf_nf_enable_defrag(struct bpf_nf_link *link)
> > >                         goto out_v6;
> > >                 }
> > >
> > > +               rcu_read_unlock();
> > >                 err = v6_hook->enable(link->net);
> > >                 if (err)
> > >                         module_put(v6_hook->owner);
> > > +
> > > +               return err;
> > >  out_v6:
> > >                 rcu_read_unlock();
> > >                 return err;
> > > @@ -114,11 +120,11 @@ static void bpf_nf_disable_defrag(struct bpf_nf_link *link)
> > >         case NFPROTO_IPV4:
> > >                 rcu_read_lock();
> > >                 v4_hook = rcu_dereference(nf_defrag_v4_hook);
> > > +               rcu_read_unlock();
> > >                 if (v4_hook) {
> > >                         v4_hook->disable(link->net);
> > >                         module_put(v4_hook->owner);
> > >                 }
> > > -               rcu_read_unlock();
> > >
> > >                 break;
> > >  #endif
> > > @@ -126,11 +132,11 @@ static void bpf_nf_disable_defrag(struct bpf_nf_link *link)
> > >         case NFPROTO_IPV6:
> > >                 rcu_read_lock();
> > >                 v6_hook = rcu_dereference(nf_defrag_v6_hook);
> > > +               rcu_read_unlock();
> >
> > No. v6_hook is gone as soon as you unlock it.
>
> I think we're protected here by the try_module_get() on the enable path.
> And we only disable defrag if enabling succeeds. The module shouldn't
> be able to deregister its hooks until we call the module_put() later.
>
> I think READ_ONCE() would've been more appropriate but I wasn't sure if
> that was ok given nf_defrag_v(4|6)_hook is written to by
> rcu_assign_pointer() and I was assuming symmetry is necessary.

Why is rcu_assign_pointer() used?
If it's not RCU protected, what is the point of rcu_*() accessors
and rcu_read_lock() ?

In general, the pattern:
rcu_read_lock();
ptr = rcu_dereference(...);
rcu_read_unlock();
ptr->..
is a bug. 100%.
Daniel Xu July 13, 2023, 11:42 p.m. UTC | #6 
On Thu, Jul 13, 2023 at 04:10:03PM -0700, Alexei Starovoitov wrote:
> On Wed, Jul 12, 2023 at 9:33 PM Daniel Xu <dxu@dxuuu.xyz> wrote:
> >
> > On Wed, Jul 12, 2023 at 06:26:13PM -0700, Alexei Starovoitov wrote:
> > > On Wed, Jul 12, 2023 at 6:22 PM Daniel Xu <dxu@dxuuu.xyz> wrote:
> > > >
> > > > Hi Alexei,
> > > >
> > > > On Wed, Jul 12, 2023 at 05:43:49PM -0700, Alexei Starovoitov wrote:
> > > > > On Wed, Jul 12, 2023 at 4:44 PM Daniel Xu <dxu@dxuuu.xyz> wrote:
> > > > > > +#if IS_ENABLED(CONFIG_NF_DEFRAG_IPV6)
> > > > > > +       case NFPROTO_IPV6:
> > > > > > +               rcu_read_lock();
> > > > > > +               v6_hook = rcu_dereference(nf_defrag_v6_hook);
> > > > > > +               if (!v6_hook) {
> > > > > > +                       rcu_read_unlock();
> > > > > > +                       err = request_module("nf_defrag_ipv6");
> > > > > > +                       if (err)
> > > > > > +                               return err < 0 ? err : -EINVAL;
> > > > > > +
> > > > > > +                       rcu_read_lock();
> > > > > > +                       v6_hook = rcu_dereference(nf_defrag_v6_hook);
> > > > > > +                       if (!v6_hook) {
> > > > > > +                               WARN_ONCE(1, "nf_defrag_ipv6_hooks bad registration");
> > > > > > +                               err = -ENOENT;
> > > > > > +                               goto out_v6;
> > > > > > +                       }
> > > > > > +               }
> > > > > > +
> > > > > > +               err = v6_hook->enable(link->net);
> > > > >
> > > > > I was about to apply, but luckily caught this issue in my local test:
> > > > >
> > > > > [   18.462448] BUG: sleeping function called from invalid context at
> > > > > kernel/locking/mutex.c:283
> > > > > [   18.463238] in_atomic(): 0, irqs_disabled(): 0, non_block: 0, pid:
> > > > > 2042, name: test_progs
> > > > > [   18.463927] preempt_count: 0, expected: 0
> > > > > [   18.464249] RCU nest depth: 1, expected: 0
> > > > > [   18.464631] CPU: 15 PID: 2042 Comm: test_progs Tainted: G
> > > > > O       6.4.0-04319-g6f6ec4fa00dc #4896
> > > > > [   18.465480] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> > > > > BIOS rel-1.12.0-59-gc9ba5276e321-prebuilt.qemu.org 04/01/2014
> > > > > [   18.466531] Call Trace:
> > > > > [   18.466767]  <TASK>
> > > > > [   18.466975]  dump_stack_lvl+0x32/0x40
> > > > > [   18.467325]  __might_resched+0x129/0x180
> > > > > [   18.467691]  mutex_lock+0x1a/0x40
> > > > > [   18.468057]  nf_defrag_ipv4_enable+0x16/0x70
> > > > > [   18.468467]  bpf_nf_link_attach+0x141/0x300
> > > > > [   18.468856]  __sys_bpf+0x133e/0x26d0
> > > > >
> > > > > You cannot call mutex under rcu_read_lock.
> > > >
> > > > Whoops, my bad. I think this patch should fix it:
> > > >
> > > > ```
> > > > From 7e8927c44452db07ddd7cf0e30bb49215fc044ed Mon Sep 17 00:00:00 2001
> > > > Message-ID: <7e8927c44452db07ddd7cf0e30bb49215fc044ed.1689211250.git.dxu@dxuuu.xyz>
> > > > From: Daniel Xu <dxu@dxuuu.xyz>
> > > > Date: Wed, 12 Jul 2023 19:17:35 -0600
> > > > Subject: [PATCH] netfilter: bpf: Don't hold rcu_read_lock during
> > > >  enable/disable
> > > >
> > > > ->enable()/->disable() takes a mutex which can sleep. You can't sleep
> > > > during RCU read side critical section.
> > > >
> > > > Our refcnt on the module will protect us from ->enable()/->disable()
> > > > from going away while we call it.
> > > >
> > > > Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
> > > > ---
> > > >  net/netfilter/nf_bpf_link.c | 10 ++++++++--
> > > >  1 file changed, 8 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/net/netfilter/nf_bpf_link.c b/net/netfilter/nf_bpf_link.c
> > > > index 77ffbf26ba3d..79704cc596aa 100644
> > > > --- a/net/netfilter/nf_bpf_link.c
> > > > +++ b/net/netfilter/nf_bpf_link.c
> > > > @@ -60,9 +60,12 @@ static int bpf_nf_enable_defrag(struct bpf_nf_link *link)
> > > >                         goto out_v4;
> > > >                 }
> > > >
> > > > +               rcu_read_unlock();
> > > >                 err = v4_hook->enable(link->net);
> > > >                 if (err)
> > > >                         module_put(v4_hook->owner);
> > > > +
> > > > +               return err;
> > > >  out_v4:
> > > >                 rcu_read_unlock();
> > > >                 return err;
> > > > @@ -92,9 +95,12 @@ static int bpf_nf_enable_defrag(struct bpf_nf_link *link)
> > > >                         goto out_v6;
> > > >                 }
> > > >
> > > > +               rcu_read_unlock();
> > > >                 err = v6_hook->enable(link->net);
> > > >                 if (err)
> > > >                         module_put(v6_hook->owner);
> > > > +
> > > > +               return err;
> > > >  out_v6:
> > > >                 rcu_read_unlock();
> > > >                 return err;
> > > > @@ -114,11 +120,11 @@ static void bpf_nf_disable_defrag(struct bpf_nf_link *link)
> > > >         case NFPROTO_IPV4:
> > > >                 rcu_read_lock();
> > > >                 v4_hook = rcu_dereference(nf_defrag_v4_hook);
> > > > +               rcu_read_unlock();
> > > >                 if (v4_hook) {
> > > >                         v4_hook->disable(link->net);
> > > >                         module_put(v4_hook->owner);
> > > >                 }
> > > > -               rcu_read_unlock();
> > > >
> > > >                 break;
> > > >  #endif
> > > > @@ -126,11 +132,11 @@ static void bpf_nf_disable_defrag(struct bpf_nf_link *link)
> > > >         case NFPROTO_IPV6:
> > > >                 rcu_read_lock();
> > > >                 v6_hook = rcu_dereference(nf_defrag_v6_hook);
> > > > +               rcu_read_unlock();
> > >
> > > No. v6_hook is gone as soon as you unlock it.
> >
> > I think we're protected here by the try_module_get() on the enable path.
> > And we only disable defrag if enabling succeeds. The module shouldn't
> > be able to deregister its hooks until we call the module_put() later.
> >
> > I think READ_ONCE() would've been more appropriate but I wasn't sure if
> > that was ok given nf_defrag_v(4|6)_hook is written to by
> > rcu_assign_pointer() and I was assuming symmetry is necessary.
> 
> Why is rcu_assign_pointer() used?
> If it's not RCU protected, what is the point of rcu_*() accessors
> and rcu_read_lock() ?
> 
> In general, the pattern:
> rcu_read_lock();
> ptr = rcu_dereference(...);
> rcu_read_unlock();
> ptr->..
> is a bug. 100%.
> 

The reason I left it like this is b/c otherwise I think there is a race
with module unload and taking a refcnt. For example:

ptr = READ_ONCE(global_var)
                                             <module unload on other cpu>
// ptr invalid
try_module_get(ptr->owner) 

I think the the synchronize_rcu() call in
kernel/module/main.c:free_module() protects against that race based on
my reading.

Maybe the ->enable() path can store a copy of the hook ptr in
struct bpf_nf_link to get rid of the odd rcu_dereference()?

Open to other ideas too -- would appreciate any hints.

Thanks,
Daniel
Florian Westphal July 14, 2023, 9:47 a.m. UTC | #7 
Daniel Xu <dxu@dxuuu.xyz> wrote:
> On Thu, Jul 13, 2023 at 04:10:03PM -0700, Alexei Starovoitov wrote:
> > Why is rcu_assign_pointer() used?
> > If it's not RCU protected, what is the point of rcu_*() accessors
> > and rcu_read_lock() ?
> > 
> > In general, the pattern:
> > rcu_read_lock();
> > ptr = rcu_dereference(...);
> > rcu_read_unlock();
> > ptr->..
> > is a bug. 100%.

FWIW, I agree with Alexei, it does look... dodgy.

> The reason I left it like this is b/c otherwise I think there is a race
> with module unload and taking a refcnt. For example:
> 
> ptr = READ_ONCE(global_var)
>                                              <module unload on other cpu>
> // ptr invalid
> try_module_get(ptr->owner) 
>

Yes, I agree.

> I think the the synchronize_rcu() call in
> kernel/module/main.c:free_module() protects against that race based on
> my reading.
> 
> Maybe the ->enable() path can store a copy of the hook ptr in
> struct bpf_nf_link to get rid of the odd rcu_dereference()?
> 
> Open to other ideas too -- would appreciate any hints.

I would suggest the following:

- Switch ordering of patches 2 and 3.
  What is currently patch 3 would add the .owner fields only.

Then, what is currently patch #2 would document the rcu/modref
interaction like this (omitting error checking for brevity):

rcu_read_lock();
v6_hook = rcu_dereference(nf_defrag_v6_hook);
if (!v6_hook) {
        rcu_read_unlock();
        err = request_module("nf_defrag_ipv6");
        if (err)
                 return err < 0 ? err : -EINVAL;
        rcu_read_lock();
	v6_hook = rcu_dereference(nf_defrag_v6_hook);
}

if (v6_hook && try_module_get(v6_hook->owner))
	v6_hook = rcu_pointer_handoff(v6_hook);
else
	v6_hook = NULL;

rcu_read_unlock();

if (!v6_hook)
	err();
v6_hook->enable();


I'd store the v4/6_hook pointer in the nf bpf link struct, its probably more
self-explanatory for the disable side in that we did pick up a module reference
that we still own at delete time, without need for any rcu involvement.

Because above handoff is repetitive for ipv4 and ipv6,
I suggest to add an agnostic helper for this.

I know you added distinct structures for ipv4 and ipv6 but if they would use
 the same one you could add

static const struct nf_defrag_hook *get_proto_frag_hook(const struct nf_defrag_hook __rcu *hook,
							const char *modulename);

And then use it like:

v4_hook = get_proto_frag_hook(nf_defrag_v4_hook, "nf_defrag_ipv4");

Without a need to copy the modprobe and handoff part.

What do you think?
Daniel Xu July 18, 2023, 9:45 p.m. UTC | #8 
Hi Florian,

On Fri, Jul 14, 2023 at 11:47:41AM +0200, Florian Westphal wrote:
> Daniel Xu <dxu@dxuuu.xyz> wrote:
> > On Thu, Jul 13, 2023 at 04:10:03PM -0700, Alexei Starovoitov wrote:
> > > Why is rcu_assign_pointer() used?
> > > If it's not RCU protected, what is the point of rcu_*() accessors
> > > and rcu_read_lock() ?
> > > 
> > > In general, the pattern:
> > > rcu_read_lock();
> > > ptr = rcu_dereference(...);
> > > rcu_read_unlock();
> > > ptr->..
> > > is a bug. 100%.
> 
> FWIW, I agree with Alexei, it does look... dodgy.
> 
> > The reason I left it like this is b/c otherwise I think there is a race
> > with module unload and taking a refcnt. For example:
> > 
> > ptr = READ_ONCE(global_var)
> >                                              <module unload on other cpu>
> > // ptr invalid
> > try_module_get(ptr->owner) 
> >
> 
> Yes, I agree.
> 
> > I think the the synchronize_rcu() call in
> > kernel/module/main.c:free_module() protects against that race based on
> > my reading.
> > 
> > Maybe the ->enable() path can store a copy of the hook ptr in
> > struct bpf_nf_link to get rid of the odd rcu_dereference()?
> > 
> > Open to other ideas too -- would appreciate any hints.
> 
> I would suggest the following:
> 
> - Switch ordering of patches 2 and 3.
>   What is currently patch 3 would add the .owner fields only.
> 
> Then, what is currently patch #2 would document the rcu/modref
> interaction like this (omitting error checking for brevity):
> 
> rcu_read_lock();
> v6_hook = rcu_dereference(nf_defrag_v6_hook);
> if (!v6_hook) {
>         rcu_read_unlock();
>         err = request_module("nf_defrag_ipv6");
>         if (err)
>                  return err < 0 ? err : -EINVAL;
>         rcu_read_lock();
> 	v6_hook = rcu_dereference(nf_defrag_v6_hook);
> }
> 
> if (v6_hook && try_module_get(v6_hook->owner))
> 	v6_hook = rcu_pointer_handoff(v6_hook);
> else
> 	v6_hook = NULL;
> 
> rcu_read_unlock();
> 
> if (!v6_hook)
> 	err();
> v6_hook->enable();
> 
> 
> I'd store the v4/6_hook pointer in the nf bpf link struct, its probably more
> self-explanatory for the disable side in that we did pick up a module reference
> that we still own at delete time, without need for any rcu involvement.
> 
> Because above handoff is repetitive for ipv4 and ipv6,
> I suggest to add an agnostic helper for this.
> 
> I know you added distinct structures for ipv4 and ipv6 but if they would use
>  the same one you could add
> 
> static const struct nf_defrag_hook *get_proto_frag_hook(const struct nf_defrag_hook __rcu *hook,
> 							const char *modulename);
> 
> And then use it like:
> 
> v4_hook = get_proto_frag_hook(nf_defrag_v4_hook, "nf_defrag_ipv4");
> 
> Without a need to copy the modprobe and handoff part.
> 
> What do you think?

That sounds reasonable to me. I'll give it a shot. Thanks for the input!

Daniel
diff mbox series

Patch

diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 600d0caebbd8..c820076c38db 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1180,6 +1180,11 @@  enum bpf_perf_event_type {
  */
 #define BPF_F_KPROBE_MULTI_RETURN	(1U << 0)
 
+/* link_create.netfilter.flags used in LINK_CREATE command for
+ * BPF_PROG_TYPE_NETFILTER to enable IP packet defragmentation.
+ */
+#define BPF_F_NETFILTER_IP_DEFRAG (1U << 0)
+
 /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
  * the following extensions:
  *
diff --git a/net/netfilter/nf_bpf_link.c b/net/netfilter/nf_bpf_link.c
index c36da56d756f..5b72aa246577 100644
--- a/net/netfilter/nf_bpf_link.c
+++ b/net/netfilter/nf_bpf_link.c
@@ -1,6 +1,7 @@ 
 // SPDX-License-Identifier: GPL-2.0
 #include <linux/bpf.h>
 #include <linux/filter.h>
+#include <linux/kmod.h>
 #include <linux/netfilter.h>
 
 #include <net/netfilter/nf_bpf_link.h>
@@ -23,8 +24,98 @@  struct bpf_nf_link {
 	struct nf_hook_ops hook_ops;
 	struct net *net;
 	u32 dead;
+	bool defrag;
 };
 
+static int bpf_nf_enable_defrag(struct bpf_nf_link *link)
+{
+	const struct nf_defrag_v4_hook __maybe_unused *v4_hook;
+	const struct nf_defrag_v6_hook __maybe_unused *v6_hook;
+	int err;
+
+	switch (link->hook_ops.pf) {
+#if IS_ENABLED(CONFIG_NF_DEFRAG_IPV4)
+	case NFPROTO_IPV4:
+		rcu_read_lock();
+		v4_hook = rcu_dereference(nf_defrag_v4_hook);
+		if (!v4_hook) {
+			rcu_read_unlock();
+			err = request_module("nf_defrag_ipv4");
+			if (err)
+				return err < 0 ? err : -EINVAL;
+
+			rcu_read_lock();
+			v4_hook = rcu_dereference(nf_defrag_v4_hook);
+			if (!v4_hook) {
+				WARN_ONCE(1, "nf_defrag_ipv4 bad registration");
+				err = -ENOENT;
+				goto out_v4;
+			}
+		}
+
+		err = v4_hook->enable(link->net);
+out_v4:
+		rcu_read_unlock();
+		return err;
+#endif
+#if IS_ENABLED(CONFIG_NF_DEFRAG_IPV6)
+	case NFPROTO_IPV6:
+		rcu_read_lock();
+		v6_hook = rcu_dereference(nf_defrag_v6_hook);
+		if (!v6_hook) {
+			rcu_read_unlock();
+			err = request_module("nf_defrag_ipv6");
+			if (err)
+				return err < 0 ? err : -EINVAL;
+
+			rcu_read_lock();
+			v6_hook = rcu_dereference(nf_defrag_v6_hook);
+			if (!v6_hook) {
+				WARN_ONCE(1, "nf_defrag_ipv6_hooks bad registration");
+				err = -ENOENT;
+				goto out_v6;
+			}
+		}
+
+		err = v6_hook->enable(link->net);
+out_v6:
+		rcu_read_unlock();
+		return err;
+#endif
+	default:
+		return -EAFNOSUPPORT;
+	}
+}
+
+static void bpf_nf_disable_defrag(struct bpf_nf_link *link)
+{
+	const struct nf_defrag_v4_hook __maybe_unused *v4_hook;
+	const struct nf_defrag_v6_hook __maybe_unused *v6_hook;
+
+	switch (link->hook_ops.pf) {
+#if IS_ENABLED(CONFIG_NF_DEFRAG_IPV4)
+	case NFPROTO_IPV4:
+		rcu_read_lock();
+		v4_hook = rcu_dereference(nf_defrag_v4_hook);
+		if (v4_hook)
+			v4_hook->disable(link->net);
+		rcu_read_unlock();
+
+		break;
+#endif
+#if IS_ENABLED(CONFIG_NF_DEFRAG_IPV6)
+	case NFPROTO_IPV6:
+		rcu_read_lock();
+		v6_hook = rcu_dereference(nf_defrag_v6_hook);
+		if (v6_hook)
+			v6_hook->disable(link->net);
+		rcu_read_unlock();
+
+		break;
+	}
+#endif
+}
+
 static void bpf_nf_link_release(struct bpf_link *link)
 {
 	struct bpf_nf_link *nf_link = container_of(link, struct bpf_nf_link, link);
@@ -37,6 +128,9 @@  static void bpf_nf_link_release(struct bpf_link *link)
 	 */
 	if (!cmpxchg(&nf_link->dead, 0, 1))
 		nf_unregister_net_hook(nf_link->net, &nf_link->hook_ops);
+
+	if (nf_link->defrag)
+		bpf_nf_disable_defrag(nf_link);
 }
 
 static void bpf_nf_link_dealloc(struct bpf_link *link)
@@ -92,6 +186,8 @@  static const struct bpf_link_ops bpf_nf_link_lops = {
 
 static int bpf_nf_check_pf_and_hooks(const union bpf_attr *attr)
 {
+	int prio;
+
 	switch (attr->link_create.netfilter.pf) {
 	case NFPROTO_IPV4:
 	case NFPROTO_IPV6:
@@ -102,19 +198,18 @@  static int bpf_nf_check_pf_and_hooks(const union bpf_attr *attr)
 		return -EAFNOSUPPORT;
 	}
 
-	if (attr->link_create.netfilter.flags)
+	if (attr->link_create.netfilter.flags & ~BPF_F_NETFILTER_IP_DEFRAG)
 		return -EOPNOTSUPP;
 
-	/* make sure conntrack confirm is always last.
-	 *
-	 * In the future, if userspace can e.g. request defrag, then
-	 * "defrag_requested && prio before NF_IP_PRI_CONNTRACK_DEFRAG"
-	 * should fail.
-	 */
-	switch (attr->link_create.netfilter.priority) {
-	case NF_IP_PRI_FIRST: return -ERANGE; /* sabotage_in and other warts */
-	case NF_IP_PRI_LAST: return -ERANGE; /* e.g. conntrack confirm */
-	}
+	/* make sure conntrack confirm is always last */
+	prio = attr->link_create.netfilter.priority;
+	if (prio == NF_IP_PRI_FIRST)
+		return -ERANGE;  /* sabotage_in and other warts */
+	else if (prio == NF_IP_PRI_LAST)
+		return -ERANGE;  /* e.g. conntrack confirm */
+	else if ((attr->link_create.netfilter.flags & BPF_F_NETFILTER_IP_DEFRAG) &&
+		 prio <= NF_IP_PRI_CONNTRACK_DEFRAG)
+		return -ERANGE;  /* cannot use defrag if prog runs before nf_defrag */
 
 	return 0;
 }
@@ -156,6 +251,18 @@  int bpf_nf_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
 		return err;
 	}
 
+	if (attr->link_create.netfilter.flags & BPF_F_NETFILTER_IP_DEFRAG) {
+		err = bpf_nf_enable_defrag(link);
+		if (err) {
+			bpf_link_cleanup(&link_primer);
+			return err;
+		}
+		/* only mark defrag enabled if enabling succeeds so cleanup path
+		 * doesn't disable without a corresponding enable
+		 */
+		link->defrag = true;
+	}
+
 	err = nf_register_net_hook(net, &link->hook_ops);
 	if (err) {
 		bpf_link_cleanup(&link_primer);
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 600d0caebbd8..c820076c38db 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -1180,6 +1180,11 @@  enum bpf_perf_event_type {
  */
 #define BPF_F_KPROBE_MULTI_RETURN	(1U << 0)
 
+/* link_create.netfilter.flags used in LINK_CREATE command for
+ * BPF_PROG_TYPE_NETFILTER to enable IP packet defragmentation.
+ */
+#define BPF_F_NETFILTER_IP_DEFRAG (1U << 0)
+
 /* When BPF ldimm64's insn[0].src_reg != 0 then this can have
  * the following extensions:
  *