| Message ID | 20230228093206.821563-1-jolsa@kernel.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | mm/bpf/perf: Store build id in inode object | expand |
On Tue, Feb 28, 2023 at 10:31:57AM +0100, Jiri Olsa wrote: > hi, > this is RFC patchset for adding build id under inode's object. > > The main change to previous post [1] is to use inode object instead of file > object for build id data. Please explain what a "build id" is, the use case for it, why we need to store it in VFS objects, what threat model it is protecting the system against, etc. > > However.. ;-) while using inode as build id storage place saves some memory > by keeping just one copy of the build id for all file instances, there seems > to be another problem. Yes, the problem being that we can cache hundreds of millions of inodes in memory, and only a very small subset of them are going to have open files associated with them. And an even smaller subset are going to be mmapped. So, in reality, this proposal won't save any memory at all - it costs memory for every inode that is not currently being used as a mmapped elf executable, right? > The problem is that we read the build id when the file is mmap-ed. Why? I'm completely clueless as to what this thing does or how it's used.... > Which is fine for our use case, Which is? -Dave.
Em Wed, Mar 01, 2023 at 09:07:14AM +1100, Dave Chinner escreveu: > On Tue, Feb 28, 2023 at 10:31:57AM +0100, Jiri Olsa wrote: > > this is RFC patchset for adding build id under inode's object. > > The main change to previous post [1] is to use inode object instead of file > > object for build id data. > > Please explain what a "build id" is, the use case for it, why we > need to store it in VFS objects, what threat model it is protecting > the system against, etc. [root@quaco ~]# file /bin/bash /bin/bash: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=160df51238a38ca27d03290f3ad5f7df75560ae0, for GNU/Linux 3.2.0, stripped [root@quaco ~]# file /lib64/libc.so.6 /lib64/libc.so.6: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=8257ee907646e9b057197533d1e4ac8ede7a9c5c, for GNU/Linux 3.2.0, not stripped [root@quaco ~]# Those BuildID[sha1]= bits, that is present in all binaries I think in all distros for quite a while. This page, from when this was initially designed, has a discussion about it, why it is needed, etc: https://fedoraproject.org/wiki/RolandMcGrath/BuildID 'perf record' will receive MMAP records, initially without build-ids, now we have one that has, but collecting it when the mmap is executed (and thus a PERF_RECORD_MMAP* record is emitted) may not work, thus this work from Jiri. - Arnaldo > > > > However.. ;-) while using inode as build id storage place saves some memory > > by keeping just one copy of the build id for all file instances, there seems > > to be another problem. > Yes, the problem being that we can cache hundreds of millions of > inodes in memory, and only a very small subset of them are going to > have open files associated with them. And an even smaller subset are > going to be mmapped. > So, in reality, this proposal won't save any memory at all - it > costs memory for every inode that is not currently being used as > a mmapped elf executable, right? > > > The problem is that we read the build id when the file is mmap-ed. > > Why? I'm completely clueless as to what this thing does or how it's > used.... > > > Which is fine for our use case, > > Which is? > > -Dave. > -- > Dave Chinner > david@fromorbit.com
On Wed, Mar 01, 2023 at 09:07:14AM +1100, Dave Chinner wrote: > On Tue, Feb 28, 2023 at 10:31:57AM +0100, Jiri Olsa wrote: > > hi, > > this is RFC patchset for adding build id under inode's object. > > > > The main change to previous post [1] is to use inode object instead of file > > object for build id data. > > Please explain what a "build id" is, the use case for it, why we > need to store it in VFS objects, what threat model it is protecting > the system against, etc. hum I still did not get your email from mailing list, just saw it from Arnaldo's reply and downloaded it from lore our use case is for hubble/tetragon [1] and we are asked to report buildid of executed binary.. but the monitoring process is running in its own pod and can't access the the binaries outside of it, so we need to be able to read it in kernel we want to read build id from BPF program attached to sched_exec tracepoint, and from BPF iterator we considered adding BPF helper and then kfunc for that, but it turned out it'd be usefull for other use cases (like retrieving build id from atomic context [2]) to have the build id stored in file (or inode) object [1] https://github.com/cilium/tetragon/ [2] https://lore.kernel.org/bpf/CA+khW7juLEcrTOd7iKG3C_WY8L265XKNo0iLzV1fE=o-cyeHcQ@mail.gmail.com/ > > > > > However.. ;-) while using inode as build id storage place saves some memory > > by keeping just one copy of the build id for all file instances, there seems > > to be another problem. > > Yes, the problem being that we can cache hundreds of millions of > inodes in memory, and only a very small subset of them are going to > have open files associated with them. And an even smaller subset are > going to be mmapped. ok, file seems like better option now > > So, in reality, this proposal won't save any memory at all - it > costs memory for every inode that is not currently being used as > a mmapped elf executable, right? right > > > The problem is that we read the build id when the file is mmap-ed. > > Why? I'm completely clueless as to what this thing does or how it's > used.... we need the build id only when the file is mmap-ed, so it seemed like the best way to read it when the file is mmaped > > > Which is fine for our use case, > > Which is? please see above thanks, jirka
On Wed, Mar 01, 2023 at 12:41:20PM -0300, Arnaldo Carvalho de Melo wrote: > Em Wed, Mar 01, 2023 at 09:07:14AM +1100, Dave Chinner escreveu: > > On Tue, Feb 28, 2023 at 10:31:57AM +0100, Jiri Olsa wrote: > > > this is RFC patchset for adding build id under inode's object. > > > > The main change to previous post [1] is to use inode object instead of file > > > object for build id data. > > > > Please explain what a "build id" is, the use case for it, why we > > need to store it in VFS objects, what threat model it is protecting > > the system against, etc. > > [root@quaco ~]# file /bin/bash > /bin/bash: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=160df51238a38ca27d03290f3ad5f7df75560ae0, for GNU/Linux 3.2.0, stripped > [root@quaco ~]# file /lib64/libc.so.6 > /lib64/libc.so.6: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=8257ee907646e9b057197533d1e4ac8ede7a9c5c, for GNU/Linux 3.2.0, not stripped > [root@quaco ~]# > > Those BuildID[sha1]= bits, that is present in all binaries I think in > all distros for quite a while. > > This page, from when this was initially designed, has a discussion about > it, why it is needed, etc: > > https://fedoraproject.org/wiki/RolandMcGrath/BuildID > > 'perf record' will receive MMAP records, initially without build-ids, > now we have one that has, but collecting it when the mmap is executed > (and thus a PERF_RECORD_MMAP* record is emitted) may not work, thus this > work from Jiri. thanks for the pointers build id is unique id for binary that's been used to identify correct binary version for related stuff.. like binary's debuginfo in perf or match binary with stack trace entries in bpf stackmap jirka > > - Arnaldo > > > > > > > However.. ;-) while using inode as build id storage place saves some memory > > > by keeping just one copy of the build id for all file instances, there seems > > > to be another problem. > > > Yes, the problem being that we can cache hundreds of millions of > > inodes in memory, and only a very small subset of them are going to > > have open files associated with them. And an even smaller subset are > > going to be mmapped. > > > So, in reality, this proposal won't save any memory at all - it > > costs memory for every inode that is not currently being used as > > a mmapped elf executable, right? > > > > > The problem is that we read the build id when the file is mmap-ed. > > > > Why? I'm completely clueless as to what this thing does or how it's > > used.... > > > > > Which is fine for our use case, > > > > Which is? > > > > -Dave. > > -- > > Dave Chinner > > david@fromorbit.com
hi, this is RFC patchset for adding build id under inode's object. The main change to previous post [1] is to use inode object instead of file object for build id data. However.. ;-) while using inode as build id storage place saves some memory by keeping just one copy of the build id for all file instances, there seems to be another problem. The problem is that we read the build id when the file is mmap-ed. Which is fine for our use case, because we only access build id data through vma->vm_file->f_inode. But there are possible scenarios/windows where the build id can be wrong when accessed in another way. Like when the file is overwritten with another binary version with different build id. This will result in having wrong build id data in inode until the new file is mmap-ed. - file open > inode->i_build_id == NULL - file mmap -> read build id > inode->i_build_id == build_id_1 [ file changed with same inode, inode keeps old i_build_id data ] - file open > inode->i_build_id == build_id_1 - file mmap -> read build id > inode->i_build_id == build_id_2 I guess we could release i_build_id when the last file's vma go out? But I'm not sure how to solve this and still be able to access build id easily just by accessing the inode->i_build_id field without any lock. I'm inclined to go back and store build id under the file object, where the build id would be correct (or missing). thoughts? thanks, jirka v2 changes: - store build id under inode [Matthew Wilcox] - use urandom_read and liburandom_read.so for test [Andrii] - add libelf-based helper to fetch build ID from elf [Andrii] - store build id or error we got when reading it [Andrii] - use full name i_build_id [Andrii] - several tests fixes [Andrii] [1] https://lore.kernel.org/bpf/20230201135737.800527-2-jolsa@kernel.org/ --- Jiri Olsa (9): mm: Store build id in inode object bpf: Use file's inode object build id in stackmap perf: Use file object build id in perf_event_mmap_event libbpf: Allow to resolve binary path in current directory selftests/bpf: Add read_buildid function selftests/bpf: Add err.h header selftests/bpf: Replace extract_build_id with read_build_id selftests/bpf: Add inode_build_id test selftests/bpf: Add iter_task_vma_buildid test fs/inode.c | 12 +++++++++++ include/linux/buildid.h | 15 ++++++++++++++ include/linux/fs.h | 7 +++++++ kernel/bpf/stackmap.c | 24 +++++++++++++++++++++- kernel/events/core.c | 46 +++++++++++++++++++++++++++++++++++++---- lib/buildid.c | 40 ++++++++++++++++++++++++++++++++++++ mm/Kconfig | 8 ++++++++ mm/mmap.c | 23 +++++++++++++++++++++ tools/lib/bpf/libbpf.c | 4 +++- tools/testing/selftests/bpf/prog_tests/bpf_iter.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/testing/selftests/bpf/prog_tests/inode_build_id.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/testing/selftests/bpf/prog_tests/stacktrace_build_id.c | 19 +++++++---------- tools/testing/selftests/bpf/prog_tests/stacktrace_build_id_nmi.c | 17 ++++++--------- tools/testing/selftests/bpf/progs/bpf_iter_task_vma_buildid.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/testing/selftests/bpf/progs/err.h | 13 ++++++++++++ tools/testing/selftests/bpf/progs/inode_build_id.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/testing/selftests/bpf/progs/profiler.inc.h | 3 +-- tools/testing/selftests/bpf/test_progs.c | 25 ---------------------- tools/testing/selftests/bpf/test_progs.h | 11 +++++++++- tools/testing/selftests/bpf/trace_helpers.c | 98 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/testing/selftests/bpf/trace_helpers.h | 5 +++++ 21 files changed, 581 insertions(+), 57 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/inode_build_id.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_task_vma_buildid.c create mode 100644 tools/testing/selftests/bpf/progs/err.h create mode 100644 tools/testing/selftests/bpf/progs/inode_build_id.c