diff mbox series

[bpf-next] bpf: dup xlated insns with kvmalloc+memcpy

Message ID 20221216055436.4698-1-sunhao.th@gmail.com (mailing list archive)
State Changes Requested
Delegated to: BPF
Headers show
Series [bpf-next] bpf: dup xlated insns with kvmalloc+memcpy | expand

Checks

Context Check Description
netdev/tree_selection success Clearly marked for bpf-next
netdev/fixes_present success Fixes tag not required for -next series
netdev/subject_prefix success Link
netdev/cover_letter success Single patches do not need cover letters
netdev/patch_count success Link
netdev/header_inline success No static functions without inline keyword in header files
netdev/build_32bit success Errors and warnings before: 2 this patch: 2
netdev/cc_maintainers success CCed 12 of 12 maintainers
netdev/build_clang success Errors and warnings before: 1 this patch: 1
netdev/module_param success Was 0 now: 0
netdev/verify_signedoff success Signed-off-by tag matches author and committer
netdev/check_selftest success No net selftest shell script
netdev/verify_fixes success No Fixes tag
netdev/build_allmodconfig_warn success Errors and warnings before: 2 this patch: 2
netdev/checkpatch success total: 0 errors, 0 warnings, 0 checks, 21 lines checked
netdev/kdoc success Errors and warnings before: 0 this patch: 0
netdev/source_inline success Was 0 now: 0
bpf/vmtest-bpf-next-VM_Test-1 success Logs for ShellCheck
bpf/vmtest-bpf-next-VM_Test-5 success Logs for build for x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-6 success Logs for build for x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-7 success Logs for llvm-toolchain
bpf/vmtest-bpf-next-VM_Test-8 success Logs for set-matrix
bpf/vmtest-bpf-next-VM_Test-2 success Logs for build for aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-3 success Logs for build for aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-4 success Logs for build for s390x with gcc
bpf/vmtest-bpf-next-VM_Test-9 success Logs for test_maps on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-10 success Logs for test_maps on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-12 success Logs for test_maps on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-13 success Logs for test_maps on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-14 success Logs for test_progs on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-15 fail Logs for test_progs on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-17 success Logs for test_progs on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-18 success Logs for test_progs on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-19 success Logs for test_progs_no_alu32 on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-20 success Logs for test_progs_no_alu32 on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-22 success Logs for test_progs_no_alu32 on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-23 success Logs for test_progs_no_alu32 on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-24 success Logs for test_progs_no_alu32_parallel on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-25 success Logs for test_progs_no_alu32_parallel on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-27 success Logs for test_progs_no_alu32_parallel on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-28 success Logs for test_progs_no_alu32_parallel on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-29 success Logs for test_progs_parallel on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-30 success Logs for test_progs_parallel on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-32 success Logs for test_progs_parallel on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-33 success Logs for test_progs_parallel on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-34 success Logs for test_verifier on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-35 success Logs for test_verifier on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-36 success Logs for test_verifier on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-37 success Logs for test_verifier on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-38 success Logs for test_verifier on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-26 success Logs for test_progs_no_alu32_parallel on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-16 success Logs for test_progs on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-21 success Logs for test_progs_no_alu32 on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-31 success Logs for test_progs_parallel on s390x with gcc
bpf/vmtest-bpf-next-PR fail PR summary
bpf/vmtest-bpf-next-VM_Test-11 success Logs for test_maps on s390x with gcc

Commit Message

Hao Sun Dec. 16, 2022, 5:54 a.m. UTC 
Currently, kmemdup() is used for allocating and copying xlated insns
in bpf_insn_prepare_dump(). The following warning can be triggered
when dup large amount of insns (roughly BPF_COMPLEXITY_LIMIT_INSNS/2)
because kmemdup() uses kmalloc() which would fail when allocing size
is too big, leading to failure in dump xlated insns:

WARNING: CPU: 2 PID: 7060 at mm/page_alloc.c:5534
Call Trace:
 <TASK>
 __alloc_pages_node include/linux/gfp.h:237 [inline]
 alloc_pages_node include/linux/gfp.h:260 [inline]
 __kmalloc_large_node+0x81/0x160 mm/slab_common.c:1096
 __do_kmalloc_node mm/slab_common.c:943 [inline]
 __kmalloc_node_track_caller.cold+0x5/0x5d mm/slab_common.c:975
 kmemdup+0x29/0x60 mm/util.c:129
 kmemdup include/linux/fortify-string.h:585 [inline]
 bpf_insn_prepare_dump kernel/bpf/syscall.c:3820 [inline]
 bpf_prog_get_info_by_fd+0x9a3/0x2cb0 kernel/bpf/syscall.c:3975
 bpf_obj_get_info_by_fd kernel/bpf/syscall.c:4297 [inline]
 __sys_bpf+0x3928/0x56f0 kernel/bpf/syscall.c:5004
 __do_sys_bpf kernel/bpf/syscall.c:5069 [inline]
 __se_sys_bpf kernel/bpf/syscall.c:5067 [inline]
 ...

So use kvmalloc()+memcpy() to fix this, for small size of insns,
this is same as kmemdup(), but this also support dup large amount
of xlated insns.

Signed-off-by: Hao Sun <sunhao.th@gmail.com>
---
 kernel/bpf/syscall.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)


base-commit: 0e43662e61f2569500ab83b8188c065603530785

Comments

Yonghong Song Dec. 16, 2022, 7:03 a.m. UTC | #1 
On 12/15/22 9:54 PM, Hao Sun wrote:
> Currently, kmemdup() is used for allocating and copying xlated insns
> in bpf_insn_prepare_dump(). The following warning can be triggered
> when dup large amount of insns (roughly BPF_COMPLEXITY_LIMIT_INSNS/2)
> because kmemdup() uses kmalloc() which would fail when allocing size
> is too big, leading to failure in dump xlated insns:
> 
> WARNING: CPU: 2 PID: 7060 at mm/page_alloc.c:5534
> Call Trace:
>   <TASK>
>   __alloc_pages_node include/linux/gfp.h:237 [inline]
>   alloc_pages_node include/linux/gfp.h:260 [inline]
>   __kmalloc_large_node+0x81/0x160 mm/slab_common.c:1096
>   __do_kmalloc_node mm/slab_common.c:943 [inline]
>   __kmalloc_node_track_caller.cold+0x5/0x5d mm/slab_common.c:975
>   kmemdup+0x29/0x60 mm/util.c:129
>   kmemdup include/linux/fortify-string.h:585 [inline]
>   bpf_insn_prepare_dump kernel/bpf/syscall.c:3820 [inline]
>   bpf_prog_get_info_by_fd+0x9a3/0x2cb0 kernel/bpf/syscall.c:3975
>   bpf_obj_get_info_by_fd kernel/bpf/syscall.c:4297 [inline]
>   __sys_bpf+0x3928/0x56f0 kernel/bpf/syscall.c:5004
>   __do_sys_bpf kernel/bpf/syscall.c:5069 [inline]
>   __se_sys_bpf kernel/bpf/syscall.c:5067 [inline]
>   ...
> 
> So use kvmalloc()+memcpy() to fix this, for small size of insns,
> this is same as kmemdup(), but this also support dup large amount
> of xlated insns.
> 
> Signed-off-by: Hao Sun <sunhao.th@gmail.com>
> ---
>   kernel/bpf/syscall.c | 8 ++++----
>   1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> index 35972afb6850..06229fddac0d 100644
> --- a/kernel/bpf/syscall.c
> +++ b/kernel/bpf/syscall.c
> @@ -3831,10 +3831,10 @@ static struct bpf_insn *bpf_insn_prepare_dump(const struct bpf_prog *prog,
>   	u8 code;
>   	int i;
>   
> -	insns = kmemdup(prog->insnsi, bpf_prog_insn_size(prog),
> -			GFP_USER);

Does kmemdup(prog->insnsi, bpf_prog_insn_size(prog), GFP_USER | 
__GFP_NOWARN) work?

> -	if (!insns)
> +	insns = kvmalloc(bpf_prog_insn_size(prog), GFP_USER | __GFP_NOWARN);
> +	if (unlikely(!insns))
>   		return insns;
> +	memcpy(insns, prog->insnsi, bpf_prog_insn_size(prog));
>   
>   	for (i = 0; i < prog->len; i++) {
>   		code = insns[i].code;
> @@ -3992,7 +3992,7 @@ static int bpf_prog_get_info_by_fd(struct file *file,
>   		uinsns = u64_to_user_ptr(info.xlated_prog_insns);
>   		ulen = min_t(u32, info.xlated_prog_len, ulen);
>   		fault = copy_to_user(uinsns, insns_sanitized, ulen);
> -		kfree(insns_sanitized);
> +		kvfree(insns_sanitized);
>   		if (fault)
>   			return -EFAULT;
>   	}
> 
> base-commit: 0e43662e61f2569500ab83b8188c065603530785
Hao Sun Dec. 16, 2022, 7:18 a.m. UTC | #2 
> On 16 Dec 2022, at 3:03 PM, Yonghong Song <yhs@meta.com> wrote:
> 
> 
> 
> On 12/15/22 9:54 PM, Hao Sun wrote:
>> Currently, kmemdup() is used for allocating and copying xlated insns
>> in bpf_insn_prepare_dump(). The following warning can be triggered
>> when dup large amount of insns (roughly BPF_COMPLEXITY_LIMIT_INSNS/2)
>> because kmemdup() uses kmalloc() which would fail when allocing size
>> is too big, leading to failure in dump xlated insns:
>> WARNING: CPU: 2 PID: 7060 at mm/page_alloc.c:5534
>> Call Trace:
>>  <TASK>
>>  __alloc_pages_node include/linux/gfp.h:237 [inline]
>>  alloc_pages_node include/linux/gfp.h:260 [inline]
>>  __kmalloc_large_node+0x81/0x160 mm/slab_common.c:1096
>>  __do_kmalloc_node mm/slab_common.c:943 [inline]
>>  __kmalloc_node_track_caller.cold+0x5/0x5d mm/slab_common.c:975
>>  kmemdup+0x29/0x60 mm/util.c:129
>>  kmemdup include/linux/fortify-string.h:585 [inline]
>>  bpf_insn_prepare_dump kernel/bpf/syscall.c:3820 [inline]
>>  bpf_prog_get_info_by_fd+0x9a3/0x2cb0 kernel/bpf/syscall.c:3975
>>  bpf_obj_get_info_by_fd kernel/bpf/syscall.c:4297 [inline]
>>  __sys_bpf+0x3928/0x56f0 kernel/bpf/syscall.c:5004
>>  __do_sys_bpf kernel/bpf/syscall.c:5069 [inline]
>>  __se_sys_bpf kernel/bpf/syscall.c:5067 [inline]
>>  ...
>> So use kvmalloc()+memcpy() to fix this, for small size of insns,
>> this is same as kmemdup(), but this also support dup large amount
>> of xlated insns.
>> Signed-off-by: Hao Sun <sunhao.th@gmail.com>
>> ---
>>  kernel/bpf/syscall.c | 8 ++++----
>>  1 file changed, 4 insertions(+), 4 deletions(-)
>> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
>> index 35972afb6850..06229fddac0d 100644
>> --- a/kernel/bpf/syscall.c
>> +++ b/kernel/bpf/syscall.c
>> @@ -3831,10 +3831,10 @@ static struct bpf_insn *bpf_insn_prepare_dump(const struct bpf_prog *prog,
>>   u8 code;
>>   int i;
>>  - insns = kmemdup(prog->insnsi, bpf_prog_insn_size(prog),
>> - GFP_USER);
> 
> Does kmemdup(prog->insnsi, bpf_prog_insn_size(prog), GFP_USER | __GFP_NOWARN) work?

This only suppress the warning, bpf_insn_prepare_dump() still fails because of
the failure of kmalloc() invoked by kmemdup(). 

> 
>> - if (!insns)
>> + insns = kvmalloc(bpf_prog_insn_size(prog), GFP_USER | __GFP_NOWARN);
>> + if (unlikely(!insns))
>>   return insns;
>> + memcpy(insns, prog->insnsi, bpf_prog_insn_size(prog));
>>     for (i = 0; i < prog->len; i++) {
>>   code = insns[i].code;
>> @@ -3992,7 +3992,7 @@ static int bpf_prog_get_info_by_fd(struct file *file,
>>   uinsns = u64_to_user_ptr(info.xlated_prog_insns);
>>   ulen = min_t(u32, info.xlated_prog_len, ulen);
>>   fault = copy_to_user(uinsns, insns_sanitized, ulen);
>> - kfree(insns_sanitized);
>> + kvfree(insns_sanitized);
>>   if (fault)
>>   return -EFAULT;
>>   }
>> base-commit: 0e43662e61f2569500ab83b8188c065603530785
Daniel Borkmann Dec. 16, 2022, 3:24 p.m. UTC | #3 
On 12/16/22 8:18 AM, Hao Sun wrote:
> 
> 
>> On 16 Dec 2022, at 3:03 PM, Yonghong Song <yhs@meta.com> wrote:
>>
>>
>>
>> On 12/15/22 9:54 PM, Hao Sun wrote:
>>> Currently, kmemdup() is used for allocating and copying xlated insns
>>> in bpf_insn_prepare_dump(). The following warning can be triggered
>>> when dup large amount of insns (roughly BPF_COMPLEXITY_LIMIT_INSNS/2)
>>> because kmemdup() uses kmalloc() which would fail when allocing size
>>> is too big, leading to failure in dump xlated insns:
>>> WARNING: CPU: 2 PID: 7060 at mm/page_alloc.c:5534
>>> Call Trace:
>>>   <TASK>
>>>   __alloc_pages_node include/linux/gfp.h:237 [inline]
>>>   alloc_pages_node include/linux/gfp.h:260 [inline]
>>>   __kmalloc_large_node+0x81/0x160 mm/slab_common.c:1096
>>>   __do_kmalloc_node mm/slab_common.c:943 [inline]
>>>   __kmalloc_node_track_caller.cold+0x5/0x5d mm/slab_common.c:975
>>>   kmemdup+0x29/0x60 mm/util.c:129
>>>   kmemdup include/linux/fortify-string.h:585 [inline]
>>>   bpf_insn_prepare_dump kernel/bpf/syscall.c:3820 [inline]
>>>   bpf_prog_get_info_by_fd+0x9a3/0x2cb0 kernel/bpf/syscall.c:3975
>>>   bpf_obj_get_info_by_fd kernel/bpf/syscall.c:4297 [inline]
>>>   __sys_bpf+0x3928/0x56f0 kernel/bpf/syscall.c:5004
>>>   __do_sys_bpf kernel/bpf/syscall.c:5069 [inline]
>>>   __se_sys_bpf kernel/bpf/syscall.c:5067 [inline]
>>>   ...
>>> So use kvmalloc()+memcpy() to fix this, for small size of insns,
>>> this is same as kmemdup(), but this also support dup large amount
>>> of xlated insns.
>>> Signed-off-by: Hao Sun <sunhao.th@gmail.com>
>>> ---
>>>   kernel/bpf/syscall.c | 8 ++++----
>>>   1 file changed, 4 insertions(+), 4 deletions(-)
>>> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
>>> index 35972afb6850..06229fddac0d 100644
>>> --- a/kernel/bpf/syscall.c
>>> +++ b/kernel/bpf/syscall.c
>>> @@ -3831,10 +3831,10 @@ static struct bpf_insn *bpf_insn_prepare_dump(const struct bpf_prog *prog,
>>>    u8 code;
>>>    int i;
>>>   - insns = kmemdup(prog->insnsi, bpf_prog_insn_size(prog),
>>> - GFP_USER);
>>
>> Does kmemdup(prog->insnsi, bpf_prog_insn_size(prog), GFP_USER | __GFP_NOWARN) work?
> 
> This only suppress the warning, bpf_insn_prepare_dump() still fails because of
> the failure of kmalloc() invoked by kmemdup().

Ok, instead of open coding, would be nice if we add a helper to mm/util.c :

void *kvmemdup(const void *src, size_t len, gfp_t gfp)
{
         void *p;

         p = kvmalloc(len, gfp);
         if (p)
                 memcpy(p, src, len);
         return p;
}
EXPORT_SYMBOL(kvmemdup);

And then bpf and in future others could make use of it.

Thanks,
Daniel
diff mbox series

Patch

diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 35972afb6850..06229fddac0d 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -3831,10 +3831,10 @@  static struct bpf_insn *bpf_insn_prepare_dump(const struct bpf_prog *prog,
 	u8 code;
 	int i;
 
-	insns = kmemdup(prog->insnsi, bpf_prog_insn_size(prog),
-			GFP_USER);
-	if (!insns)
+	insns = kvmalloc(bpf_prog_insn_size(prog), GFP_USER | __GFP_NOWARN);
+	if (unlikely(!insns))
 		return insns;
+	memcpy(insns, prog->insnsi, bpf_prog_insn_size(prog));
 
 	for (i = 0; i < prog->len; i++) {
 		code = insns[i].code;
@@ -3992,7 +3992,7 @@  static int bpf_prog_get_info_by_fd(struct file *file,
 		uinsns = u64_to_user_ptr(info.xlated_prog_insns);
 		ulen = min_t(u32, info.xlated_prog_len, ulen);
 		fault = copy_to_user(uinsns, insns_sanitized, ulen);
-		kfree(insns_sanitized);
+		kvfree(insns_sanitized);
 		if (fault)
 			return -EFAULT;
 	}