| Message ID | 20200521165641.15940-1-kristen@linux.intel.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Function Granular KASLR | expand |
On Thu, May 21, 2020 at 09:56:31AM -0700, Kristen Carlson Accardi wrote: > Changes in v2: > -------------- > * Fix to address i386 build failure > * Allow module reordering patch to be configured separately so that > arm (or other non-x86_64 arches) can take advantage of module function > reordering. This support has not be tested by me, but smoke tested by > Ard Biesheuvel <ardb@kernel.org> on arm. > * Fix build issue when building on arm as reported by > Ard Biesheuvel <ardb@kernel.org> > * minor chages for certain checkpatch warnings and review feedback. I successfully built and booted this on top of linux-next. For my builds I include: CONFIG_LOCK_DEBUGGING_SUPPORT=y CONFIG_PROVE_LOCKING=y CONFIG_DEBUG_RT_MUTEXES=y CONFIG_DEBUG_SPINLOCK=y CONFIG_DEBUG_MUTEXES=y CONFIG_DEBUG_WW_MUTEX_SLOWPATH=y CONFIG_DEBUG_RWSEMS=y CONFIG_DEBUG_LOCK_ALLOC=y CONFIG_LOCKDEP=y CONFIG_DEBUG_ATOMIC_SLEEP=y which catches various things. One of those (I assume either CONFIG_LOCKDEP or CONFIG_DEBUG_MUTEXES) has found an issue with kallsyms: [ 34.112989] ------------[ cut here ]------------ [ 34.113560] WARNING: CPU: 1 PID: 1997 at kernel/module.c:260 module_assert_mutex+0x29/0x30 [ 34.114479] Modules linked in: [ 34.114831] CPU: 1 PID: 1997 Comm: grep Tainted: G W 5.7.0-rc6-next-20200519+ #497 ... [ 34.128556] Call Trace: [ 34.128867] module_kallsyms_on_each_symbol+0x1d/0xa0 [ 34.130238] kallsyms_on_each_symbol+0xbd/0xd0 [ 34.131642] kallsyms_sorted_open+0x3f/0x70 [ 34.132160] proc_reg_open+0x99/0x180 [ 34.133222] do_dentry_open+0x176/0x400 [ 34.134182] vfs_open+0x2d/0x30 [ 34.134579] do_open.isra.0+0x2a0/0x410 [ 34.135058] path_openat+0x175/0x620 [ 34.135506] do_filp_open+0x91/0x100 [ 34.136912] do_sys_openat2+0x210/0x2d0 [ 34.137388] do_sys_open+0x46/0x80 [ 34.137818] __x64_sys_openat+0x20/0x30 [ 34.138288] do_syscall_64+0x55/0x1d0 [ 34.138720] entry_SYSCALL_64_after_hwframe+0x49/0xb3 Triggering it is easy, just "cat /proc/kallsyms" (and I'd note that I don't even have any modules loaded). Tracking this down, it just looks like kallsyms needs to hold a lock while sorting: diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index 558963b275ec..182b16a6079b 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -772,7 +772,9 @@ static int kallsyms_sorted_open(struct inode *inode, struct file *file) INIT_LIST_HEAD(list); + mutex_lock(&module_mutex); ret = kallsyms_on_each_symbol(get_all_symbol_name, list); + mutex_unlock(&module_mutex); if (ret != 0) return ret; This fixes it for me. Everything else seems to be lovely. :) Nice work!
Kristen, Kristen Carlson Accardi <kristen@linux.intel.com> writes: sorry for not following this work and a maybe stupid question. > Proposed Improvement > -------------------- > This patch set proposes adding function reordering on top of the existing > KASLR base address randomization. The over-arching objective is incremental > improvement over what we already have. It is designed to work in combination > with the existing solution. The implementation is really pretty simple, and > there are 2 main area where changes occur: > > * Build time > > GCC has had an option to place functions into individual .text sections for > many years now. This option can be used to implement function reordering at > load time. The final compiled vmlinux retains all the section headers, which > can be used to help find the address ranges of each function. Using this > information and an expanded table of relocation addresses, individual text > sections can be suffled immediately after decompression. Some data tables > inside the kernel that have assumptions about order require re-sorting > after being updated when applying relocations. In order to modify these tables, > a few key symbols are excluded from the objcopy symbol stripping process for > use after shuffling the text segments. I understand how this is supposed to work, but I fail to find an explanation how all of this is preserving the text subsections we have, i.e. .kprobes.text, .entry.text ...? I assume that the functions in these subsections are reshuffled within their own randomized address space so that __xxx_text_start and __xxx_text_end markers still make sense, right? I'm surely too tired to figure it out from the patches, but you really want to explain that very detailed for mere mortals who are not deep into this magic as you are. Thanks, tglx
On Fri, May 22, 2020 at 12:26:30AM +0200, Thomas Gleixner wrote: > I understand how this is supposed to work, but I fail to find an > explanation how all of this is preserving the text subsections we have, > i.e. .kprobes.text, .entry.text ...? I had the same question when I first started looking at earlier versions of this series! :) > I assume that the functions in these subsections are reshuffled within > their own randomized address space so that __xxx_text_start and > __xxx_text_end markers still make sense, right? No, but perhaps in the future. Right now, they are entirely ignored and left untouched. The current series only looks at the sections produced by -ffunction-sections, which is to say only things named ".text.$thing" (e.g. ".text.func1", ".text.func2"). Since the "special" text sections in the kernel are named ".$thing.text" (specifically to avoid other long-standing linker logic that does similar .text.* pattern matches) they get ignored by FGKASLR right now too. Even more specifically, they're ignored because all of these special _input_ sections are actually manually collected by the linker script into the ".text" _output_ section, which FGKASLR ignores -- it can only randomize the final output sections (and has no basic block visibility into the section contents), so everything in .text is untouched. Because these special sections are collapsed into the single .text output section is why we've needed the __$thing_start and __$thing_end symbols manually constructed by the linker scripts: we lose input section location/size details once the linker collects them into an output section. > I'm surely too tired to figure it out from the patches, but you really > want to explain that very detailed for mere mortals who are not deep > into this magic as you are. Yeah, it's worth calling out, especially since it's an area of future work -- I think if we can move the special sections out of .text into their own output sections that can get randomized and we'll have section position/size information available without the manual ..._start/_end symbols. But this will require work with the compiler and linker to get what's needed relative to -ffunction-sections, teach the kernel about the new way of getting _start/_end, etc etc. So, before any of that, just .text.* is a good first step, and after that I think next would be getting .text randomized relative to the other .text.* sections (IIUC, it is entirely untouched currently, so only the standard KASLR base offset moves it around). Only after that do we start poking around trying to munge the special section contents (which requires use solving a few problems simultaneously). :)
Kees, Kees Cook <keescook@chromium.org> writes: > On Fri, May 22, 2020 at 12:26:30AM +0200, Thomas Gleixner wrote: >> I understand how this is supposed to work, but I fail to find an >> explanation how all of this is preserving the text subsections we have, >> i.e. .kprobes.text, .entry.text ...? > > I had the same question when I first started looking at earlier versions > of this series! :) > >> I assume that the functions in these subsections are reshuffled within >> their own randomized address space so that __xxx_text_start and >> __xxx_text_end markers still make sense, right? > > No, but perhaps in the future. Right now, they are entirely ignored and > left untouched. I'm fine with that restriction, but for a moment I got worried that this might screw up explicit subsections. This really want's to be clearly expressed in the cover letter and the changelogs so that such questions don't arise again. <SNIP> > So, before any of that, just .text.* is a good first step, and after > that I think next would be getting .text randomized relative to the other > .text.* sections (IIUC, it is entirely untouched currently, so only the > standard KASLR base offset moves it around). Only after that do we start > poking around trying to munge the special section contents (which > requires use solving a few problems simultaneously). :) Thanks for the detailed explanation! tglx
On Thu, 2020-05-21 at 16:30 -0700, Kees Cook wrote: > On Fri, May 22, 2020 at 12:26:30AM +0200, Thomas Gleixner wrote: > > I understand how this is supposed to work, but I fail to find an > > explanation how all of this is preserving the text subsections we > > have, > > i.e. .kprobes.text, .entry.text ...? > > I had the same question when I first started looking at earlier > versions > of this series! :) Thanks for responding - clearly I do need to update the cover letter and documentation. > > > I assume that the functions in these subsections are reshuffled > > within > > their own randomized address space so that __xxx_text_start and > > __xxx_text_end markers still make sense, right? > > No, but perhaps in the future. Right now, they are entirely ignored > and > left untouched. The current series only looks at the sections > produced > by -ffunction-sections, which is to say only things named > ".text.$thing" > (e.g. ".text.func1", ".text.func2"). Since the "special" text > sections > in the kernel are named ".$thing.text" (specifically to avoid other > long-standing linker logic that does similar .text.* pattern matches) > they get ignored by FGKASLR right now too. > > Even more specifically, they're ignored because all of these special > _input_ sections are actually manually collected by the linker script > into the ".text" _output_ section, which FGKASLR ignores -- it can > only > randomize the final output sections (and has no basic block > visibility > into the section contents), so everything in .text is untouched. > Because > these special sections are collapsed into the single .text output > section is why we've needed the __$thing_start and __$thing_end > symbols > manually constructed by the linker scripts: we lose input section > location/size details once the linker collects them into an output > section. > > > I'm surely too tired to figure it out from the patches, but you > > really > > want to explain that very detailed for mere mortals who are not > > deep > > into this magic as you are. > > Yeah, it's worth calling out, especially since it's an area of future > work -- I think if we can move the special sections out of .text into > their own output sections that can get randomized and we'll have > section > position/size information available without the manual ..._start/_end > symbols. But this will require work with the compiler and linker to > get > what's needed relative to -ffunction-sections, teach the kernel about > the new way of getting _start/_end, etc etc. > > So, before any of that, just .text.* is a good first step, and after > that I think next would be getting .text randomized relative to the > other > .text.* sections (IIUC, it is entirely untouched currently, so only > the > standard KASLR base offset moves it around). Only after that do we > start > poking around trying to munge the special section contents (which > requires use solving a few problems simultaneously). :) > That's right - we keep .text unrandomized, so any special sections that are collected into .text are still in their original layout. Like you said, they still get to take advantage of normal KASLR (base address randomization).