| Message ID | Zl4AHfG6Gg5Htdgc@x1 (mailing list archive) |
|---|---|
| State | Not Applicable |
| Headers | show |
| Series | elfutils DWARF problem was: Re: Problem with BTF generation on mips64el | expand |
| Context | Check | Description |
|---|---|---|
| netdev/tree_selection | success | Not a local patch |
Hi, On Mon, Jun 03, 2024 at 02:40:45PM -0300, Arnaldo Carvalho de Melo wrote: > Couldn't find a way to ask eu-readelf for more verbose output, where we > could perhaps get some clue as to why it produces nothing while binutils > readelf manages to grok it, Mark, do you know some other way to ask > eu-readelf to produce more debug output? > > I'm unsure if the netdevsim.ko file was left in a semi encoded BTF state > that then made eu-readelf to not be able to process it while pahole, > that uses eltuils' libraries, was able to process the first two CUs for > a kernel module and all the CUs for the vmlinux file :-\ I haven't looked at the vmlinux file. But for the .ko file the issue is that the elfutils MIPS backend isn't complete. Specifically MIPS relocations aren't recognized (and so cannot be applied). There are some pending patches which try to fix that: https://patchwork.sourceware.org/project/elfutils/list/?series=31601 Cheers, Mark
Hi Mark, On Mon, Jun 03, 2024 at 09:18:33PM +0200, Mark Wielaard wrote: > On Mon, Jun 03, 2024 at 02:40:45PM -0300, Arnaldo Carvalho de Melo wrote: > > Couldn't find a way to ask eu-readelf for more verbose output, where we > > could perhaps get some clue as to why it produces nothing while binutils > > readelf manages to grok it, Mark, do you know some other way to ask > > eu-readelf to produce more debug output? > > > > I'm unsure if the netdevsim.ko file was left in a semi encoded BTF state > > that then made eu-readelf to not be able to process it while pahole, > > that uses eltuils' libraries, was able to process the first two CUs for > > a kernel module and all the CUs for the vmlinux file :-\ > > > > Mark, the whole thread is available at: > > > > https://lore.kernel.org/all/Zl3Zp5r9m6X_i_J4@x1/T/#u > > I haven't looked at the vmlinux file. But for the .ko file the issue > is that the elfutils MIPS backend isn't complete. Specifically MIPS > relocations aren't recognized (and so cannot be applied). There are > some pending patches which try to fix that: > > https://patchwork.sourceware.org/project/elfutils/list/?series=31601 Earlier in the thread, Hengqi Chen pointed out the latest elfutils backend work for MIPS, and I locally rebuilt elfutils and then pahole from their respective next/main branches. For elfutils, main (935ee131cf7c) includes e259f126 Support Mips architecture f2acb069 stack: Fix stack unwind failure on mips db33cb0c backends: Add register_info, return_value_location, core_note mips which partially applies the patchwork series but leaves out the support for readelf, strip, and elflint. I believe this means the vmlinux and .ko files I shared are OK, or is there more backend work needed for MIPS? The bits missing in eu-readelf would explain the blank output both Arnaldo and I see from "$ eu-readelf -winfo vmlinux". I tried rebuilding with the patchwork readelf patch locally but ran into merge conflicts. CCing Ying Huang for any more insight. Thanks, Tony
Hi Tony, Yes, I also found that merge will have some conflicts now. And you can modify the key code as follows to fix the issue about "eu-readelf -w" could not show info. But this can not fix other display error in the result of the "eu-readelf -w" which need modify the way to get symbol index and type. > diff --git a/libelf/libelfP.h b/libelf/libelfP.h > index bdd2cc6a..6565ee02 100644 > --- a/libelf/libelfP.h > +++ b/libelf/libelfP.h > @@ -620,4 +620,5 @@ extern void __libelf_reset_rawdata (Elf_Scn *scn, void *buf, size_t size, > #define ELF64_MIPS_R_TYPE1(i) ((i) & 0xff) > #define ELF64_MIPS_R_TYPE2(i) (((i) >> 8) & 0xff) > #define ELF64_MIPS_R_TYPE3(i) (((i) >> 16) & 0xff) > +#define is_debug_section_type(type) (type == SHT_PROGBITS || type == SHT_MIPS_DWARF) > #endif /* libelfP.h */ > diff --git a/src/readelf.c b/src/readelf.c > index 0e931184..e88cf67c 100644 > --- a/src/readelf.c > +++ b/src/readelf.c > @@ -12043,7 +12139,7 @@ print_debug (Dwfl_Module *dwflmod, Ebl *ebl, GElf_Ehdr *ehdr) > GElf_Shdr shdr_mem; > GElf_Shdr *shdr = gelf_getshdr (scn, &shdr_mem); > > - if (shdr != NULL && shdr->sh_type == SHT_PROGBITS) > + if (shdr != NULL && is_debug_section_type(shdr->sh_type)) > { > const char *name = elf_strptr (ebl->elf, shstrndx, > shdr->sh_name); > @@ -12073,7 +12169,7 @@ print_debug (Dwfl_Module *dwflmod, Ebl *ebl, GElf_Ehdr *ehdr) > GElf_Shdr shdr_mem; > GElf_Shdr *shdr = gelf_getshdr (scn, &shdr_mem); > > - if (shdr != NULL && shdr->sh_type == SHT_PROGBITS) > + if (shdr != NULL && is_debug_section_type(shdr->sh_type)) > { > static const struct > { Thanks, Ying 在 2024/6/4 11:47, Tony Ambardar 写道: > Hi Mark, > > On Mon, Jun 03, 2024 at 09:18:33PM +0200, Mark Wielaard wrote: >> On Mon, Jun 03, 2024 at 02:40:45PM -0300, Arnaldo Carvalho de Melo wrote: >>> Couldn't find a way to ask eu-readelf for more verbose output, where we >>> could perhaps get some clue as to why it produces nothing while binutils >>> readelf manages to grok it, Mark, do you know some other way to ask >>> eu-readelf to produce more debug output? >>> >>> I'm unsure if the netdevsim.ko file was left in a semi encoded BTF state >>> that then made eu-readelf to not be able to process it while pahole, >>> that uses eltuils' libraries, was able to process the first two CUs for >>> a kernel module and all the CUs for the vmlinux file :-\ >>> >>> Mark, the whole thread is available at: >>> >>> https://lore.kernel.org/all/Zl3Zp5r9m6X_i_J4@x1/T/#u >> I haven't looked at the vmlinux file. But for the .ko file the issue >> is that the elfutils MIPS backend isn't complete. Specifically MIPS >> relocations aren't recognized (and so cannot be applied). There are >> some pending patches which try to fix that: >> >> https://patchwork.sourceware.org/project/elfutils/list/?series=31601 > Earlier in the thread, Hengqi Chen pointed out the latest elfutils backend > work for MIPS, and I locally rebuilt elfutils and then pahole from their > respective next/main branches. For elfutils, main (935ee131cf7c) includes > > e259f126 Support Mips architecture > f2acb069 stack: Fix stack unwind failure on mips > db33cb0c backends: Add register_info, return_value_location, core_note mips > > which partially applies the patchwork series but leaves out the support for > readelf, strip, and elflint. > > I believe this means the vmlinux and .ko files I shared are OK, or is there > more backend work needed for MIPS? > > The bits missing in eu-readelf would explain the blank output both Arnaldo > and I see from "$ eu-readelf -winfo vmlinux". I tried rebuilding with the > patchwork readelf patch locally but ran into merge conflicts. > > CCing Ying Huang for any more insight. > > Thanks, > Tony
On Mon, Jun 03, 2024 at 08:47:24PM -0700, Tony Ambardar wrote: > Hi Mark, > > On Mon, Jun 03, 2024 at 09:18:33PM +0200, Mark Wielaard wrote: > > On Mon, Jun 03, 2024 at 02:40:45PM -0300, Arnaldo Carvalho de Melo wrote: > > > Couldn't find a way to ask eu-readelf for more verbose output, where we > > > could perhaps get some clue as to why it produces nothing while binutils > > > readelf manages to grok it, Mark, do you know some other way to ask > > > eu-readelf to produce more debug output? > > > > > > I'm unsure if the netdevsim.ko file was left in a semi encoded BTF state > > > that then made eu-readelf to not be able to process it while pahole, > > > that uses eltuils' libraries, was able to process the first two CUs for > > > a kernel module and all the CUs for the vmlinux file :-\ > > > > > > Mark, the whole thread is available at: > > > > > > https://lore.kernel.org/all/Zl3Zp5r9m6X_i_J4@x1/T/#u > > > > I haven't looked at the vmlinux file. But for the .ko file the issue > > is that the elfutils MIPS backend isn't complete. Specifically MIPS > > relocations aren't recognized (and so cannot be applied). There are > > some pending patches which try to fix that: > > > > https://patchwork.sourceware.org/project/elfutils/list/?series=31601 > > Earlier in the thread, Hengqi Chen pointed out the latest elfutils backend > work for MIPS, and I locally rebuilt elfutils and then pahole from their > respective next/main branches. For elfutils, main (935ee131cf7c) includes > > e259f126 Support Mips architecture > f2acb069 stack: Fix stack unwind failure on mips > db33cb0c backends: Add register_info, return_value_location, core_note mips > > which partially applies the patchwork series but leaves out the support for > readelf, strip, and elflint. > > I believe this means the vmlinux and .ko files I shared are OK, or is there > more backend work needed for MIPS? > > The bits missing in eu-readelf would explain the blank output both Arnaldo > and I see from "$ eu-readelf -winfo vmlinux". I tried rebuilding with the > patchwork readelf patch locally but ran into merge conflicts. > > CCing Ying Huang for any more insight. > > Thanks, > Tony Hello all, A short update, starting with answering my own question. No, apparently the above commits *do not* complete the backend work. Ying Huang submitted additional related patches since March 5: [1][2] strip: Adapt src/strip -o -f on mips readelf: Adapt src/readelf -h/-S/-r/-w/-l/-d/-a on mips elflint: adapt src/elflint --gnu src/nm on mips test: Add mips in run-allregs.sh and run-readelf-mixed-corenote.sh Despite the titles, these patches do include core backend changes for MIPS. I resolved the various merge conflicts [3], rebuilt elfutils, and retested kernel builds to now find: - pahole is able to read DWARF[45] info and create .BTF for modules - resolve_btfids can successfully patch .BTF_ids in modules - kernel successfully loads modules with BTF and kfuncs (tested 6.6 LTS) Huzzah! Ying: Thank you for developing these MIPS patches. In your view, are the MIPS changes now complete, or do you plan further updates that might improve or impact parsing DWARF debug/reloc info in apps like pahole? Mark: Given that BTF usage on Linux/MIPS is basically broken without these patches, could I request some of your review time for them to be merged? If it's helpful, my branch [3] includes all patches with conflicts fixed, and I also successfully ran the elfutils self-tests (including MIPS from Ying). Please feel free to add for these patches: Tested-by: Tony Ambardar <Tony.Ambardar@gmail.com> Arnaldo: Your stepping through DWARF/reloc diagnostics earlier was helpful. Thanks! I reran your tests with the updated elfutils and latest pahole (pre-1.27), and then found: - everything that worked before, still works - your observations from "btfdiff vmlinux" and 'struct dma_chan' persist - we now see expected output from "eu-readelf -winfo netdevsim.ko" Regarding pahole, DWARF parsing and BTF generation now works: (with no more die__process: error messages seen) kodidev:~/linux$ pahole -F dwarf netdevsim.ko |wc -l 14504 but strangely pahole still doesn't read its own generated BTF: kodidev:~/linux$ pahole -F btf netdevsim.ko libbpf: Invalid BTF string section pahole: file 'netdevsim.ko' has no btf type information. Poking inside a little further: kodidev:~/linux$ ltrace -S pahole -F btf netdevsim.ko [...] argp_parse(0x563d47da42a0, 4, 0x7ffd5e552698, 0 <unfinished ...> SYS_318(0x7fc385bf84d8, 8, 1, 0x7fc385ce9908) = 8 SYS_brk(0) = 0x563d47e37000 SYS_brk(0x563d47e58000) = 0x563d47e58000 <... argp_parse resumed> ) = 0 dwarves__init(0x20000, 0, -4096, 213) = 0 dwarves__resolve_cacheline_size(0x563d47da40c0, 0, 24, 213) = 64 cus__new(5, 0, 64, 213) = 0x563d47e372a0 memset(0x563d47da44c0, ' ', 127) = 0x563d47da44c0 strlen("/sys/kernel/btf/") = 16 strncmp("netdevsim.ko", "/sys/kernel/btf/", 16) = 63 cus__load_files(0x563d47e372a0, 0x563d47da40c0, 0x7ffd5e5526b0, 0x563d47da40c0 <unfinished ...> SYS_openat(0xffffff9c, 0x7ffd5e554603, 0x80000, 0) = 3 SYS_newfstatat(3, 0x7fc385baf44f, 0x7ffd5e552210, 4096) = 0 SYS_read(3, "\177ELF\002\001\001", 4096) = 4096 SYS_close(3) = 0 SYS_openat(0xffffff9c, 0x7ffd5e554603, 0x80000, 0) = 3 SYS_fcntl(3, 1, 0, 0x7fc3859de2f0) = 1 SYS_newfstatat(3, 0x7fc385baf44f, 0x7ffd5e5521f0, 4096) = 0 SYS_pread(3, 0x7ffd5e5521f0, 64, 0) = 64 SYS_pread(3, 0x563d47e3e470, 3520, 0x4f60c8) = 3520 SYS_pread(3, 0x563d47e3f240, 525, 0x4f5eb8) = 525 SYS_pread(3, 0x563d47e3f460, 0x45dd, 0x2b1fb6) = 0x45dd SYS_write(2, "libbpf: Invalid BTF string secti"..., 35libbpf: Invalid BTF string section ) = 35 SYS_close(3) = 0 <... cus__load_files resumed> ) = 0xffffffff access("netdevsim.ko", 4 <unfinished ...> SYS_access("netdevsim.ko", 04) = 0 <... access resumed> ) = 0 fprintf(0x7fc385bf26a0, "pahole: file '%s' has no %s type"..., "netdevsim.ko", "btf" <unfinished ...> SYS_write(2, "pahole: file 'netdevsim.ko' has "..., 57pahole: file 'netdevsim.ko' has no btf type information. ) = 57 <... fprintf resumed> ) = 57 SYS_exit_group(1 <no return ...> +++ exited (status 1) +++ Could you help investigate this further? Maybe a libbpf issue? For the record, I also tried building pahole with embedded libbpf 1.4.3 without any change. (side note: please make pahole --version also cover libbpf) Many thanks everyone for your help, Tony [1]: https://patchwork.sourceware.org/project/elfutils/list/?series=31601 [2]: https://patchwork.sourceware.org/project/elfutils/list/?series=34310 [3]: https://github.com/guidosarducci/elfutils/commits/main-fix-mips-support-reloc/
On Mon, Jun 10, 2024 at 11:36:54PM -0700, Tony Ambardar wrote: > On Mon, Jun 03, 2024 at 08:47:24PM -0700, Tony Ambardar wrote: > > [snip] > > Arnaldo: > > Your stepping through DWARF/reloc diagnostics earlier was helpful. Thanks! > I reran your tests with the updated elfutils and latest pahole (pre-1.27), > and then found: > > - everything that worked before, still works > - your observations from "btfdiff vmlinux" and 'struct dma_chan' persist > - we now see expected output from "eu-readelf -winfo netdevsim.ko" > > Regarding pahole, DWARF parsing and BTF generation now works: > (with no more die__process: error messages seen) > > kodidev:~/linux$ pahole -F dwarf netdevsim.ko |wc -l > 14504 > > but strangely pahole still doesn't read its own generated BTF: > > kodidev:~/linux$ pahole -F btf netdevsim.ko > libbpf: Invalid BTF string section > pahole: file 'netdevsim.ko' has no btf type information. After staring at this I realized that this is split BTF and we needed to issue "pahole -F btf --btf_base vmlinux netdevsim.ko", which does work. Unfortunately, the libbpf error message is a bit cryptic; perhaps a future update could mention "--btf_base"? > > [snip] > > Many thanks everyone for your help, > Tony > > [1]: https://patchwork.sourceware.org/project/elfutils/list/?series=31601 > [2]: https://patchwork.sourceware.org/project/elfutils/list/?series=34310 > [3]: > https://github.com/guidosarducci/elfutils/commits/main-fix-mips-support-reloc/
Hi, Adding elfutils-devel to CC to keep everyone up to date on the state of the patches. On Mon, 2024-06-10 at 23:36 -0700, Tony Ambardar wrote: > On Mon, Jun 03, 2024 at 08:47:24PM -0700, Tony Ambardar wrote: > > On Mon, Jun 03, 2024 at 09:18:33PM +0200, Mark Wielaard wrote: > > > On Mon, Jun 03, 2024 at 02:40:45PM -0300, Arnaldo Carvalho de Melo wrote: > > > > Couldn't find a way to ask eu-readelf for more verbose output, where we > > > > could perhaps get some clue as to why it produces nothing while binutils > > > > readelf manages to grok it, Mark, do you know some other way to ask > > > > eu-readelf to produce more debug output? > > > > > > > > I'm unsure if the netdevsim.ko file was left in a semi encoded BTF state > > > > that then made eu-readelf to not be able to process it while pahole, > > > > that uses eltuils' libraries, was able to process the first two CUs for > > > > a kernel module and all the CUs for the vmlinux file :-\ > > > > > > > > Mark, the whole thread is available at: > > > > > > > > https://lore.kernel.org/all/Zl3Zp5r9m6X_i_J4@x1/T/#u > > > > > > I haven't looked at the vmlinux file. But for the .ko file the issue > > > is that the elfutils MIPS backend isn't complete. Specifically MIPS > > > relocations aren't recognized (and so cannot be applied). There are > > > some pending patches which try to fix that: > > > > > > https://patchwork.sourceware.org/project/elfutils/list/?series=31601 > > > > Earlier in the thread, Hengqi Chen pointed out the latest elfutils backend > > work for MIPS, and I locally rebuilt elfutils and then pahole from their > > respective next/main branches. For elfutils, main (935ee131cf7c) includes > > > > e259f126 Support Mips architecture > > f2acb069 stack: Fix stack unwind failure on mips > > db33cb0c backends: Add register_info, return_value_location, core_note mips > > > > which partially applies the patchwork series but leaves out the support for > > readelf, strip, and elflint. > > > > I believe this means the vmlinux and .ko files I shared are OK, or is there > > more backend work needed for MIPS? > > > > The bits missing in eu-readelf would explain the blank output both Arnaldo > > and I see from "$ eu-readelf -winfo vmlinux". I tried rebuilding with the > > patchwork readelf patch locally but ran into merge conflicts. > > A short update, starting with answering my own question. > > No, apparently the above commits *do not* complete the backend work. Ying > Huang submitted additional related patches since March 5: [1][2] > > strip: Adapt src/strip -o -f on mips > readelf: Adapt src/readelf -h/-S/-r/-w/-l/-d/-a on mips > elflint: adapt src/elflint --gnu src/nm on mips > test: Add mips in run-allregs.sh and run-readelf-mixed-corenote.sh > > Despite the titles, these patches do include core backend changes for MIPS. > I resolved the various merge conflicts [3], rebuilt elfutils, and retested > kernel builds to now find: > > - pahole is able to read DWARF[45] info and create .BTF for modules > - resolve_btfids can successfully patch .BTF_ids in modules > - kernel successfully loads modules with BTF and kfuncs (tested 6.6 LTS) > > Huzzah! > > > Ying: > > Thank you for developing these MIPS patches. In your view, are the MIPS > changes now complete, or do you plan further updates that might improve or > impact parsing DWARF debug/reloc info in apps like pahole? > > > Mark: > > Given that BTF usage on Linux/MIPS is basically broken without these > patches, could I request some of your review time for them to be merged? If > it's helpful, my branch [3] includes all patches with conflicts fixed, and > I also successfully ran the elfutils self-tests (including MIPS from Ying). > Please feel free to add for these patches: > > Tested-by: Tony Ambardar <Tony.Ambardar@gmail.com> Yes, I would very much like to integrate the rest of these patches. But I keep running out of time. The main issues were that, as you noticed, the patches mix backend and frontend tool changes a bit. I don't have access to a MIPS system to test them on. There are a couple of different MIPS abis (I believe all combinations of 32/64 bit and big/little endianness), but people have only tested on mips64le (maybe that is the only relevant one these days?) And finally the way MIPS represents relocations is slightly different than any other ELF architecture does. So we have to translate that somewhere to make the standards functions work. I have to convince myself that doing that in elf_getdata as the patches do is the right place. > Many thanks everyone for your help, > Tony > > [1]: https://patchwork.sourceware.org/project/elfutils/list/?series=31601 > [2]: https://patchwork.sourceware.org/project/elfutils/list/?series=34310 > [3]: > https://github.com/guidosarducci/elfutils/commits/main-fix-mips-support-reloc/
On Tue, Jun 11, 2024 at 03:07:29PM +0200, Mark Wielaard wrote: > Hi, > > Adding elfutils-devel to CC to keep everyone up to date on the state of > the patches. > > On Mon, 2024-06-10 at 23:36 -0700, Tony Ambardar wrote: > > On Mon, Jun 03, 2024 at 08:47:24PM -0700, Tony Ambardar wrote: > > > On Mon, Jun 03, 2024 at 09:18:33PM +0200, Mark Wielaard wrote: > > > > On Mon, Jun 03, 2024 at 02:40:45PM -0300, Arnaldo Carvalho de Melo wrote: > > > > > Couldn't find a way to ask eu-readelf for more verbose output, where we > > > > > could perhaps get some clue as to why it produces nothing while binutils > > > > > readelf manages to grok it, Mark, do you know some other way to ask > > > > > eu-readelf to produce more debug output? > > > > > > > > > > I'm unsure if the netdevsim.ko file was left in a semi encoded BTF state > > > > > that then made eu-readelf to not be able to process it while pahole, > > > > > that uses eltuils' libraries, was able to process the first two CUs for > > > > > a kernel module and all the CUs for the vmlinux file :-\ > > > > > > > > > > Mark, the whole thread is available at: > > > > > > > > > > https://lore.kernel.org/all/Zl3Zp5r9m6X_i_J4@x1/T/#u > > > > > > > > I haven't looked at the vmlinux file. But for the .ko file the issue > > > > is that the elfutils MIPS backend isn't complete. Specifically MIPS > > > > relocations aren't recognized (and so cannot be applied). There are > > > > some pending patches which try to fix that: > > > > > > > > https://patchwork.sourceware.org/project/elfutils/list/?series=31601 > > > > > > Earlier in the thread, Hengqi Chen pointed out the latest elfutils backend > > > work for MIPS, and I locally rebuilt elfutils and then pahole from their > > > respective next/main branches. For elfutils, main (935ee131cf7c) includes > > > > > > e259f126 Support Mips architecture > > > f2acb069 stack: Fix stack unwind failure on mips > > > db33cb0c backends: Add register_info, return_value_location, core_note mips > > > > > > which partially applies the patchwork series but leaves out the support for > > > readelf, strip, and elflint. > > > > > > I believe this means the vmlinux and .ko files I shared are OK, or is there > > > more backend work needed for MIPS? > > > > > > The bits missing in eu-readelf would explain the blank output both Arnaldo > > > and I see from "$ eu-readelf -winfo vmlinux". I tried rebuilding with the > > > patchwork readelf patch locally but ran into merge conflicts. > > > > A short update, starting with answering my own question. > > > > No, apparently the above commits *do not* complete the backend work. Ying > > Huang submitted additional related patches since March 5: [1][2] > > > > strip: Adapt src/strip -o -f on mips > > readelf: Adapt src/readelf -h/-S/-r/-w/-l/-d/-a on mips > > elflint: adapt src/elflint --gnu src/nm on mips > > test: Add mips in run-allregs.sh and run-readelf-mixed-corenote.sh > > > > Despite the titles, these patches do include core backend changes for MIPS. > > I resolved the various merge conflicts [3], rebuilt elfutils, and retested > > kernel builds to now find: > > > > - pahole is able to read DWARF[45] info and create .BTF for modules > > - resolve_btfids can successfully patch .BTF_ids in modules > > - kernel successfully loads modules with BTF and kfuncs (tested 6.6 LTS) > > > > Huzzah! > > > > > > Ying: > > > > Thank you for developing these MIPS patches. In your view, are the MIPS > > changes now complete, or do you plan further updates that might improve or > > impact parsing DWARF debug/reloc info in apps like pahole? > > > > > > Mark: > > > > Given that BTF usage on Linux/MIPS is basically broken without these > > patches, could I request some of your review time for them to be merged? If > > it's helpful, my branch [3] includes all patches with conflicts fixed, and > > I also successfully ran the elfutils self-tests (including MIPS from Ying). > > Please feel free to add for these patches: > > > > Tested-by: Tony Ambardar <Tony.Ambardar@gmail.com> > > Yes, I would very much like to integrate the rest of these patches. But > I keep running out of time. The main issues were that, as you noticed, > the patches mix backend and frontend tool changes a bit. I don't have > access to a MIPS system to test them on. There are a couple of > different MIPS abis (I believe all combinations of 32/64 bit and > big/little endianness), but people have only tested on mips64le (maybe > that is the only relevant one these days?) And finally the way MIPS > represents relocations is slightly different than any other ELF > architecture does. So we have to translate that somewhere to make the > standards functions work. I have to convince myself that doing that in > elf_getdata as the patches do is the right place. > Glad to hear there's strong interest. Not much I can add to the code structure discussion but I *can* confirm my testing included both mips64el and mips32be, specifically to improve word-size/endianness coverage. The former is supported by Debian, while the latter can still be found on many embedded devices like consumer routers. If you need additional testing for review/merge I can help with that (using OpenWrt/QEMU primarily). > > Many thanks everyone for your help, > > Tony > > > > [1]: https://patchwork.sourceware.org/project/elfutils/list/?series=31601 > > [2]: https://patchwork.sourceware.org/project/elfutils/list/?series=34310 > > [3]: > > https://github.com/guidosarducci/elfutils/commits/main-fix-mips-support-reloc/ >
Hi Tony, Thanks for your fix about the conflict and test about the whole patch. Now, there is currently one test case that has not been added, which is tests/run-backtrace-core-mips.sh. Thanks, Ying 在 2024/6/11 14:36, Tony Ambardar 写道: > On Mon, Jun 03, 2024 at 08:47:24PM -0700, Tony Ambardar wrote: >> Hi Mark, >> >> On Mon, Jun 03, 2024 at 09:18:33PM +0200, Mark Wielaard wrote: >>> On Mon, Jun 03, 2024 at 02:40:45PM -0300, Arnaldo Carvalho de Melo wrote: >>>> Couldn't find a way to ask eu-readelf for more verbose output, where we >>>> could perhaps get some clue as to why it produces nothing while binutils >>>> readelf manages to grok it, Mark, do you know some other way to ask >>>> eu-readelf to produce more debug output? >>>> >>>> I'm unsure if the netdevsim.ko file was left in a semi encoded BTF state >>>> that then made eu-readelf to not be able to process it while pahole, >>>> that uses eltuils' libraries, was able to process the first two CUs for >>>> a kernel module and all the CUs for the vmlinux file :-\ >>>> >>>> Mark, the whole thread is available at: >>>> >>>> https://lore.kernel.org/all/Zl3Zp5r9m6X_i_J4@x1/T/#u >>> I haven't looked at the vmlinux file. But for the .ko file the issue >>> is that the elfutils MIPS backend isn't complete. Specifically MIPS >>> relocations aren't recognized (and so cannot be applied). There are >>> some pending patches which try to fix that: >>> >>> https://patchwork.sourceware.org/project/elfutils/list/?series=31601 >> Earlier in the thread, Hengqi Chen pointed out the latest elfutils backend >> work for MIPS, and I locally rebuilt elfutils and then pahole from their >> respective next/main branches. For elfutils, main (935ee131cf7c) includes >> >> e259f126 Support Mips architecture >> f2acb069 stack: Fix stack unwind failure on mips >> db33cb0c backends: Add register_info, return_value_location, core_note mips >> >> which partially applies the patchwork series but leaves out the support for >> readelf, strip, and elflint. >> >> I believe this means the vmlinux and .ko files I shared are OK, or is there >> more backend work needed for MIPS? >> >> The bits missing in eu-readelf would explain the blank output both Arnaldo >> and I see from "$ eu-readelf -winfo vmlinux". I tried rebuilding with the >> patchwork readelf patch locally but ran into merge conflicts. >> >> CCing Ying Huang for any more insight. >> >> Thanks, >> Tony > Hello all, > > A short update, starting with answering my own question. > > No, apparently the above commits *do not* complete the backend work. Ying > Huang submitted additional related patches since March 5: [1][2] > > strip: Adapt src/strip -o -f on mips > readelf: Adapt src/readelf -h/-S/-r/-w/-l/-d/-a on mips > elflint: adapt src/elflint --gnu src/nm on mips > test: Add mips in run-allregs.sh and run-readelf-mixed-corenote.sh > > Despite the titles, these patches do include core backend changes for MIPS. > I resolved the various merge conflicts [3], rebuilt elfutils, and retested > kernel builds to now find: > > - pahole is able to read DWARF[45] info and create .BTF for modules > - resolve_btfids can successfully patch .BTF_ids in modules > - kernel successfully loads modules with BTF and kfuncs (tested 6.6 LTS) > > Huzzah! > > > Ying: > > Thank you for developing these MIPS patches. In your view, are the MIPS > changes now complete, or do you plan further updates that might improve or > impact parsing DWARF debug/reloc info in apps like pahole? > > > Mark: > > Given that BTF usage on Linux/MIPS is basically broken without these > patches, could I request some of your review time for them to be merged? If > it's helpful, my branch [3] includes all patches with conflicts fixed, and > I also successfully ran the elfutils self-tests (including MIPS from Ying). > Please feel free to add for these patches: > > Tested-by: Tony Ambardar <Tony.Ambardar@gmail.com> > > > Arnaldo: > > Your stepping through DWARF/reloc diagnostics earlier was helpful. Thanks! > I reran your tests with the updated elfutils and latest pahole (pre-1.27), > and then found: > > - everything that worked before, still works > - your observations from "btfdiff vmlinux" and 'struct dma_chan' persist > - we now see expected output from "eu-readelf -winfo netdevsim.ko" > > Regarding pahole, DWARF parsing and BTF generation now works: > (with no more die__process: error messages seen) > > kodidev:~/linux$ pahole -F dwarf netdevsim.ko |wc -l > 14504 > > but strangely pahole still doesn't read its own generated BTF: > > kodidev:~/linux$ pahole -F btf netdevsim.ko > libbpf: Invalid BTF string section > pahole: file 'netdevsim.ko' has no btf type information. > > Poking inside a little further: > > kodidev:~/linux$ ltrace -S pahole -F btf netdevsim.ko > [...] > argp_parse(0x563d47da42a0, 4, 0x7ffd5e552698, 0 <unfinished ...> > SYS_318(0x7fc385bf84d8, 8, 1, 0x7fc385ce9908) = 8 > SYS_brk(0) = 0x563d47e37000 > SYS_brk(0x563d47e58000) = 0x563d47e58000 > <... argp_parse resumed> ) = 0 > dwarves__init(0x20000, 0, -4096, 213) = 0 > dwarves__resolve_cacheline_size(0x563d47da40c0, 0, 24, 213) = 64 > cus__new(5, 0, 64, 213) = 0x563d47e372a0 > memset(0x563d47da44c0, ' ', 127) = 0x563d47da44c0 > strlen("/sys/kernel/btf/") = 16 > strncmp("netdevsim.ko", "/sys/kernel/btf/", 16) = 63 > cus__load_files(0x563d47e372a0, 0x563d47da40c0, 0x7ffd5e5526b0, > 0x563d47da40c0 <unfinished ...> > SYS_openat(0xffffff9c, 0x7ffd5e554603, 0x80000, 0) = 3 > SYS_newfstatat(3, 0x7fc385baf44f, 0x7ffd5e552210, 4096) = 0 > SYS_read(3, "\177ELF\002\001\001", 4096) = 4096 > SYS_close(3) = 0 > SYS_openat(0xffffff9c, 0x7ffd5e554603, 0x80000, 0) = 3 > SYS_fcntl(3, 1, 0, 0x7fc3859de2f0) = 1 > SYS_newfstatat(3, 0x7fc385baf44f, 0x7ffd5e5521f0, 4096) = 0 > SYS_pread(3, 0x7ffd5e5521f0, 64, 0) = 64 > SYS_pread(3, 0x563d47e3e470, 3520, 0x4f60c8) = 3520 > SYS_pread(3, 0x563d47e3f240, 525, 0x4f5eb8) = 525 > SYS_pread(3, 0x563d47e3f460, 0x45dd, 0x2b1fb6) = 0x45dd > SYS_write(2, "libbpf: Invalid BTF string secti"..., 35libbpf: Invalid BTF > string section > ) = 35 > SYS_close(3) = 0 > <... cus__load_files resumed> ) = 0xffffffff > access("netdevsim.ko", 4 <unfinished ...> > SYS_access("netdevsim.ko", 04) = 0 > <... access resumed> ) = 0 > fprintf(0x7fc385bf26a0, "pahole: file '%s' has no %s type"..., > "netdevsim.ko", "btf" <unfinished ...> > SYS_write(2, "pahole: file 'netdevsim.ko' has "..., 57pahole: file > 'netdevsim.ko' has no btf type information. > ) = 57 > <... fprintf resumed> ) = 57 > SYS_exit_group(1 <no return ...> > +++ exited (status 1) +++ > > Could you help investigate this further? Maybe a libbpf issue? For the > record, I also tried building pahole with embedded libbpf 1.4.3 without > any change. (side note: please make pahole --version also cover libbpf) > > > Many thanks everyone for your help, > Tony > > [1]: https://patchwork.sourceware.org/project/elfutils/list/?series=31601 > [2]: https://patchwork.sourceware.org/project/elfutils/list/?series=34310 > [3]: > https://github.com/guidosarducci/elfutils/commits/main-fix-mips-support-reloc/
Hi Mark, Regarding the current questions, I have a few points that needed to be explained. 在 2024/6/11 21:07, Mark Wielaard 写道: > Hi, > > Adding elfutils-devel to CC to keep everyone up to date on the state of > the patches. > > On Mon, 2024-06-10 at 23:36 -0700, Tony Ambardar wrote: >> On Mon, Jun 03, 2024 at 08:47:24PM -0700, Tony Ambardar wrote: >>> On Mon, Jun 03, 2024 at 09:18:33PM +0200, Mark Wielaard wrote: >>>> On Mon, Jun 03, 2024 at 02:40:45PM -0300, Arnaldo Carvalho de Melo wrote: >>>>> Couldn't find a way to ask eu-readelf for more verbose output, where we >>>>> could perhaps get some clue as to why it produces nothing while binutils >>>>> readelf manages to grok it, Mark, do you know some other way to ask >>>>> eu-readelf to produce more debug output? >>>>> >>>>> I'm unsure if the netdevsim.ko file was left in a semi encoded BTF state >>>>> that then made eu-readelf to not be able to process it while pahole, >>>>> that uses eltuils' libraries, was able to process the first two CUs for >>>>> a kernel module and all the CUs for the vmlinux file :-\ >>>>> >>>>> Mark, the whole thread is available at: >>>>> >>>>> https://lore.kernel.org/all/Zl3Zp5r9m6X_i_J4@x1/T/#u >>>> I haven't looked at the vmlinux file. But for the .ko file the issue >>>> is that the elfutils MIPS backend isn't complete. Specifically MIPS >>>> relocations aren't recognized (and so cannot be applied). There are >>>> some pending patches which try to fix that: >>>> >>>> https://patchwork.sourceware.org/project/elfutils/list/?series=31601 >>> Earlier in the thread, Hengqi Chen pointed out the latest elfutils backend >>> work for MIPS, and I locally rebuilt elfutils and then pahole from their >>> respective next/main branches. For elfutils, main (935ee131cf7c) includes >>> >>> e259f126 Support Mips architecture >>> f2acb069 stack: Fix stack unwind failure on mips >>> db33cb0c backends: Add register_info, return_value_location, core_note mips >>> >>> which partially applies the patchwork series but leaves out the support for >>> readelf, strip, and elflint. >>> >>> I believe this means the vmlinux and .ko files I shared are OK, or is there >>> more backend work needed for MIPS? >>> >>> The bits missing in eu-readelf would explain the blank output both Arnaldo >>> and I see from "$ eu-readelf -winfo vmlinux". I tried rebuilding with the >>> patchwork readelf patch locally but ran into merge conflicts. >> A short update, starting with answering my own question. >> >> No, apparently the above commits *do not* complete the backend work. Ying >> Huang submitted additional related patches since March 5: [1][2] >> >> strip: Adapt src/strip -o -f on mips >> readelf: Adapt src/readelf -h/-S/-r/-w/-l/-d/-a on mips >> elflint: adapt src/elflint --gnu src/nm on mips >> test: Add mips in run-allregs.sh and run-readelf-mixed-corenote.sh >> >> Despite the titles, these patches do include core backend changes for MIPS. >> I resolved the various merge conflicts [3], rebuilt elfutils, and retested >> kernel builds to now find: >> >> - pahole is able to read DWARF[45] info and create .BTF for modules >> - resolve_btfids can successfully patch .BTF_ids in modules >> - kernel successfully loads modules with BTF and kfuncs (tested 6.6 LTS) >> >> Huzzah! >> >> >> Ying: >> >> Thank you for developing these MIPS patches. In your view, are the MIPS >> changes now complete, or do you plan further updates that might improve or >> impact parsing DWARF debug/reloc info in apps like pahole? >> >> >> Mark: >> >> Given that BTF usage on Linux/MIPS is basically broken without these >> patches, could I request some of your review time for them to be merged? If >> it's helpful, my branch [3] includes all patches with conflicts fixed, and >> I also successfully ran the elfutils self-tests (including MIPS from Ying). >> Please feel free to add for these patches: >> >> Tested-by: Tony Ambardar <Tony.Ambardar@gmail.com> > Yes, I would very much like to integrate the rest of these patches. But > I keep running out of time. The main issues were that, as you noticed, > the patches mix backend and frontend tool changes a bit. The reason about the mixture and title is that only by fixing the acquisition of relocation information can 'strip' and 'readelf -w' work normally. Now it seems really confusing, so I want to do some changes based on your opinions, change the titles or reorganize these patches to make them look more logical. > I don't have > access to a MIPS system to test them on. There are a couple of > different MIPS abis (I believe all combinations of 32/64 bit and > big/little endianness), but people have only tested on mips64le (maybe > that is the only relevant one these days?) I have tested other mips abi before, I will test it agagin and attach the results of 'make check' with and without patch. And add a description of mips abi support in the commit message. > And finally the way MIPS > represents relocations is slightly different than any other ELF > architecture does. So we have to translate that somewhere to make the > standards functions work. I have to convince myself that doing that in > elf_getdata as the patches do is the right place. The controversial problem was the location of the code, the code was to change the original relocation info to ensure mips can obtain the correct index value and symble index. Or we did not modify the original data, we modify the way to obtain the index value and symble index at funtion 'gelf_getrela' in file 'libelf/gelf_getrela.c','libelf/gelf_getrel.c','libelf/gelf_update_rela.c','libelf/gelf_update_rel.c'. Where the function 'gelf_getrela' is called，modify the relocation info that has been obtained . What do you think of this? Thanks, Ying > >> Many thanks everyone for your help, >> Tony >> >> [1]: https://patchwork.sourceware.org/project/elfutils/list/?series=31601 >> [2]: https://patchwork.sourceware.org/project/elfutils/list/?series=34310 >> [3]: >> https://github.com/guidosarducci/elfutils/commits/main-fix-mips-support-reloc/
diff --git a/dwarf_loader.c b/dwarf_loader.c index f59477b44dfea026..b832c93cc2194eaf 100644 --- a/dwarf_loader.c +++ b/dwarf_loader.c @@ -2847,8 +2847,8 @@ static int die__process(Dwarf_Die *die, struct cu *cu, struct conf_load *conf) } if (tag != DW_TAG_compile_unit && tag != DW_TAG_type_unit) { - fprintf(stderr, "%s: DW_TAG_compile_unit, DW_TAG_type_unit, DW_TAG_partial_unit or DW_TAG_skeleton_unit expected got %s (0x%x)!\n", - __FUNCTION__, dwarf_tag_name(tag), tag); + fprintf(stderr, "%s: DW_TAG_compile_unit, DW_TAG_type_unit, DW_TAG_partial_unit or DW_TAG_skeleton_unit expected got %s (0x%x) @ %llx!\n", + __FUNCTION__, dwarf_tag_name(tag), tag, (unsigned long long)dwarf_dieoffset(die)); return -EINVAL; }