| Message ID | 20241001225207.2215639-1-andrii@kernel.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | uprobes,mm: speculative lockless VMA-to-uprobe lookup | expand |
Well. I am in no position to ack these changes. But let me say that I personally like this series and I see nothing wrong, except perhaps 5/5 needs some data_race/etc annotations as we have already discussed. Thanks, Oleg. On 10/01, Andrii Nakryiko wrote: > > Implement speculative (lockless) resolution of VMA to inode to uprobe, > bypassing the need to take mmap_lock for reads, if possible. Patch #1 by Suren > adds mm_struct helpers that help detect whether mm_struct were changed, which > is used by uprobe logic to validate that speculative results can be trusted > after all the lookup logic results in a valid uprobe instance. Patch #2 > follows to make mm_lock_seq into 64-bit counter (on 64-bit architectures). > > Patch #3 adds back RCU-delayed freeing for FMODE_BACKING files, which is > necessary to make speculation safe to access struct file's memory in any > possible situation. > > Patch #4 is a simplification to uprobe VMA flag checking, suggested by Oleg. > > And, finally, patch #5 is the speculative VMA-to-uprobe resolution logic. See > corresponding patch for details and benchmarking results. > > v1->v2: > - adjusted vma_end_write_all() comment to point out it should never be called > manually now, but I wasn't sure how ACQUIRE/RELEASE comments should be > reworded (previously requested by Jann), so I'd appreciate some help there > (Jann); > - int -> long change for mm_lock_seq, as agreed at LPC2024 (Jann, Suren, Liam); > - kfree_rcu_mightsleep() for FMODE_BACKING (Suren, Christian); > - vm_flags simplification in find_active_uprobe_rcu() and > find_active_uprobe_speculative() (Oleg); > - guard(rcu)() simplified find_active_uprobe_speculative() implementation. > > Andrii Nakryiko (4): > mm: switch to 64-bit mm_lock_seq/vm_lock_seq on 64-bit architectures > fs: add back RCU-delayed freeing of FMODE_BACKING file > uprobes: simplify find_active_uprobe_rcu() VMA checks > uprobes: add speculative lockless VMA-to-inode-to-uprobe resolution > > Suren Baghdasaryan (1): > mm: introduce mmap_lock_speculation_{start|end} > > fs/file_table.c | 2 +- > include/linux/mm.h | 6 ++-- > include/linux/mm_types.h | 7 ++-- > include/linux/mmap_lock.h | 72 ++++++++++++++++++++++++++++++++------- > kernel/events/uprobes.c | 46 ++++++++++++++++++++++++- > kernel/fork.c | 3 -- > 6 files changed, 114 insertions(+), 22 deletions(-) > > -- > 2.43.5 >
Implement speculative (lockless) resolution of VMA to inode to uprobe, bypassing the need to take mmap_lock for reads, if possible. Patch #1 by Suren adds mm_struct helpers that help detect whether mm_struct were changed, which is used by uprobe logic to validate that speculative results can be trusted after all the lookup logic results in a valid uprobe instance. Patch #2 follows to make mm_lock_seq into 64-bit counter (on 64-bit architectures). Patch #3 adds back RCU-delayed freeing for FMODE_BACKING files, which is necessary to make speculation safe to access struct file's memory in any possible situation. Patch #4 is a simplification to uprobe VMA flag checking, suggested by Oleg. And, finally, patch #5 is the speculative VMA-to-uprobe resolution logic. See corresponding patch for details and benchmarking results. v1->v2: - adjusted vma_end_write_all() comment to point out it should never be called manually now, but I wasn't sure how ACQUIRE/RELEASE comments should be reworded (previously requested by Jann), so I'd appreciate some help there (Jann); - int -> long change for mm_lock_seq, as agreed at LPC2024 (Jann, Suren, Liam); - kfree_rcu_mightsleep() for FMODE_BACKING (Suren, Christian); - vm_flags simplification in find_active_uprobe_rcu() and find_active_uprobe_speculative() (Oleg); - guard(rcu)() simplified find_active_uprobe_speculative() implementation. Andrii Nakryiko (4): mm: switch to 64-bit mm_lock_seq/vm_lock_seq on 64-bit architectures fs: add back RCU-delayed freeing of FMODE_BACKING file uprobes: simplify find_active_uprobe_rcu() VMA checks uprobes: add speculative lockless VMA-to-inode-to-uprobe resolution Suren Baghdasaryan (1): mm: introduce mmap_lock_speculation_{start|end} fs/file_table.c | 2 +- include/linux/mm.h | 6 ++-- include/linux/mm_types.h | 7 ++-- include/linux/mmap_lock.h | 72 ++++++++++++++++++++++++++++++++------- kernel/events/uprobes.c | 46 ++++++++++++++++++++++++- kernel/fork.c | 3 -- 6 files changed, 114 insertions(+), 22 deletions(-)