diff mbox

Re: [PATCH v2 00/29] implement KASLR for ARM

Message ID CAKv+Gu9c_j0qriRazSTmq8eD9icCu4Wuzw1z-LF7bubLVLQd7Q@mail.gmail.com (mailing list archive)
State New, archived
Headers show

Commit Message

Ard Biesheuvel Sept. 5, 2017, 7:42 p.m. UTC 
On 5 September 2017 at 20:37, Tony Lindgren <tony@atomide.com> wrote:
> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170905 09:49]:
>> On 5 September 2017 at 17:45, Tony Lindgren <tony@atomide.com> wrote:
>> > I did see boot attempts fail with randomize enable where no output
>> > was produced. It seems this is happening for me maybe 1 out of 5 boots.
>> > Enabling DEBUG_LL did not show anything either.
>> >
>>
>> Yes. I am looking into a couple of kernelci boot reports that look
>> suspicious, but it is rather difficult to reproduce, for obvious
>> reasons :-)
>>
>> Which hardware are you testing this on?
>
> For testing on omap3, I'm mostly using logicpd torpedo devkit as
> it works out of box with PM with mainline Linux and has NFSroot
> usable too.
>

Right. Well, I will try to reproduce with the BB white I have.

Are you booting with an initrd?

>> > Then loading modules with CONFIG_RANDOMIZE_BASE=y seems to fail with:
>> >
>> > $ sudo modprobe rtc-twl
>> > rtc_twl: disagrees about version of symbol module_layout
>> > modprobe: ERROR: could not insert 'rtc_twl': Exec format error
>> >
>>
>> Is this with CONFIG_MODVERSIONS enabled?
>
> Yes, but disabling that did not seem to make any difference
> based on just one test.
>

Yeah, well, it appears I missed a couple of details :-)

This should fix the module loading issues:

Comments

Tony Lindgren Sept. 5, 2017, 9:27 p.m. UTC | #1 
* Ard Biesheuvel <ard.biesheuvel@linaro.org> [170905 12:43]:
> Right. Well, I will try to reproduce with the BB white I have.

Yeah that should be reproducable, I got it to happen on BBB here
after about 5 boots.

> Are you booting with an initrd?

Not on this one, on beagleboard xm I do.

> >> > Then loading modules with CONFIG_RANDOMIZE_BASE=y seems to fail with:
> >> >
> >> > $ sudo modprobe rtc-twl
> >> > rtc_twl: disagrees about version of symbol module_layout
> >> > modprobe: ERROR: could not insert 'rtc_twl': Exec format error
> >> >
> >>
> >> Is this with CONFIG_MODVERSIONS enabled?
> >
> > Yes, but disabling that did not seem to make any difference
> > based on just one test.
> >
> 
> Yeah, well, it appears I missed a couple of details :-)
> 
> This should fix the module loading issues:

Yeah now modprobe works :) That's after manually applying
it as the tabs got munched in your patch somewhere, see below.

Regards,

Tony

> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index 1a0304dd388d..bbefd5f32ec2 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -1830,6 +1830,8 @@ config RANDOMIZE_BASE
>   depends on MMU && AUTO_ZRELADDR
>   depends on !XIP_KERNEL && !ZBOOT_ROM
>   select RELOCATABLE
> + select ARM_MODULE_PLTS if MODULES
> + select MODULE_REL_CRCS if MODVERSIONS
>   help
>    Randomizes the virtual and physical address at which the kernel
>    image is loaded, as a security feature that deters exploit attempts
> diff --git a/arch/arm/include/asm/elf.h b/arch/arm/include/asm/elf.h
> index f13ae153fb24..b56fc4dd27b6 100644
> --- a/arch/arm/include/asm/elf.h
> +++ b/arch/arm/include/asm/elf.h
> @@ -50,6 +50,7 @@ typedef struct user_fp elf_fpregset_t;
>  #define R_ARM_NONE 0
>  #define R_ARM_PC24 1
>  #define R_ARM_ABS32 2
> +#define R_ARM_REL32 3
>  #define R_ARM_CALL 28
>  #define R_ARM_JUMP24 29
>  #define R_ARM_TARGET1 38
> diff --git a/arch/arm/kernel/module.c b/arch/arm/kernel/module.c
> index 3ff571c2c71c..aa4d72837cd5 100644
> --- a/arch/arm/kernel/module.c
> +++ b/arch/arm/kernel/module.c
> @@ -175,6 +175,10 @@
>   *(u32 *)loc |= offset & 0x7fffffff;
>   break;
> 
> + case R_ARM_REL32:
> + *(u32 *)loc += sym->st_value - loc;
> + break;
> +
>   case R_ARM_MOVW_ABS_NC:
>   case R_ARM_MOVT_ABS:
>   offset = tmp = __mem_to_opcode_arm(*(u32 *)loc);
Ard Biesheuvel Sept. 5, 2017, 9:31 p.m. UTC | #2 
On 5 September 2017 at 22:27, Tony Lindgren <tony@atomide.com> wrote:
> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170905 12:43]:
>> Right. Well, I will try to reproduce with the BB white I have.
>
> Yeah that should be reproducable, I got it to happen on BBB here
> after about 5 boots.
>
>> Are you booting with an initrd?
>
> Not on this one, on beagleboard xm I do.
>

Interesting.

>> >> > Then loading modules with CONFIG_RANDOMIZE_BASE=y seems to fail with:
>> >> >
>> >> > $ sudo modprobe rtc-twl
>> >> > rtc_twl: disagrees about version of symbol module_layout
>> >> > modprobe: ERROR: could not insert 'rtc_twl': Exec format error
>> >> >
>> >>
>> >> Is this with CONFIG_MODVERSIONS enabled?
>> >
>> > Yes, but disabling that did not seem to make any difference
>> > based on just one test.
>> >
>>
>> Yeah, well, it appears I missed a couple of details :-)
>>
>> This should fix the module loading issues:
>
> Yeah now modprobe works :) That's after manually applying
> it as the tabs got munched in your patch somewhere, see below.
>

Yeah, Gmail tends to do that now and again. Apologies.

Thanks for confirming that the module loading works for you, I will
fold this into my v3.
Ard Biesheuvel Sept. 6, 2017, 10:40 a.m. UTC | #3 
On 5 September 2017 at 22:31, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> On 5 September 2017 at 22:27, Tony Lindgren <tony@atomide.com> wrote:
>> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170905 12:43]:
>>> Right. Well, I will try to reproduce with the BB white I have.
>>
>> Yeah that should be reproducable, I got it to happen on BBB here
>> after about 5 boots.
>>

Unfortunately, my BB white boots happily every time.

I have added DEBUG_UNCOMPRESS support to mach-omap2, and added some
debug output to the kaslr init routine

For example,

Starting kernel ...

regions.image_size:00e00000
regions.pa_start:80000000
regions.pa_end:90000000
regions.zimage_start:80200000
regions.zimage_size:00430228
regions.dtb_start:82000000
regions.dtb_size:00009000
num:00000016
*kaslr_offset:04400000
Uncompressing Linux... done, booting the kernel.
[    0.000000] Booting Linux on physical CPU 0x0
 ....
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xd0000000 - 0xff800000   ( 760 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xcfe00000   ( 254 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc4408000 - 0xc4d00000   (9184 kB)
[    0.000000]       .init : 0xc5000000 - 0xc5100000   (1024 kB)
[    0.000000]       .data : 0xc5100000 - 0xc5149df0   ( 296 kB)
[    0.000000]        .bss : 0xc51cb000 - 0xc59a9a8c   (8059 kB)

I have pushed the changes to my arm-kaslr-v3 branch. If you are
willing to spend more cycles on this, could you please run it with
DEBUG_LL enabled, and capture the output of the failed boots?
Tony Lindgren Sept. 6, 2017, 4:22 p.m. UTC | #4 
* Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 03:41]:
> On 5 September 2017 at 22:31, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> > On 5 September 2017 at 22:27, Tony Lindgren <tony@atomide.com> wrote:
> >> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170905 12:43]:
> >>> Right. Well, I will try to reproduce with the BB white I have.
> >>
> >> Yeah that should be reproducable, I got it to happen on BBB here
> >> after about 5 boots.
> >>
> 
> Unfortunately, my BB white boots happily every time.
> 
> I have added DEBUG_UNCOMPRESS support to mach-omap2, and added some
> debug output to the kaslr init routine

OK. FYI, the DEBUG_UNCOMPRESS support works in Linux next as
it's now using the generic 8250 debug_ll code.

> For example,
> 
> Starting kernel ...
> 
> regions.image_size:00e00000
> regions.pa_start:80000000
> regions.pa_end:90000000
> regions.zimage_start:80200000
> regions.zimage_size:00430228
> regions.dtb_start:82000000
> regions.dtb_size:00009000
> num:00000016
> *kaslr_offset:04400000
> Uncompressing Linux... done, booting the kernel.
> [    0.000000] Booting Linux on physical CPU 0x0
>  ....
> [    0.000000] Virtual kernel memory layout:
> [    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
> [    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
> [    0.000000]     vmalloc : 0xd0000000 - 0xff800000   ( 760 MB)
> [    0.000000]     lowmem  : 0xc0000000 - 0xcfe00000   ( 254 MB)
> [    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
> [    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
> [    0.000000]       .text : 0xc4408000 - 0xc4d00000   (9184 kB)
> [    0.000000]       .init : 0xc5000000 - 0xc5100000   (1024 kB)
> [    0.000000]       .data : 0xc5100000 - 0xc5149df0   ( 296 kB)
> [    0.000000]        .bss : 0xc51cb000 - 0xc59a9a8c   (8059 kB)
> 
> I have pushed the changes to my arm-kaslr-v3 branch. If you are
> willing to spend more cycles on this, could you please run it with
> DEBUG_LL enabled, and capture the output of the failed boots?

Sure was not able to reproduce it so far on BBB. But here's
failed boot output from logicpd-torpedo-37xx-devkit. Will
try some more booting on BBB too.

Regards,

Tony

8< -------------------
Kernel image @ 0x81000000 [ 0x000000 - 0x426810 ]
## Flattened Device Tree blob at 84000000
   Booting using the fdt blob at 0x84000000
   Loading Device Tree to 86feb000, end 86fff2d5 ... OK

Starting kernel ...

regions.image_size:00e00000
regions.pa_start:80000000
regions.pa_end:88000000
regions.zimage_start:81000000
regions.zimage_size:00437830
regions.dtb_start:86feb000
regions.dtb_size:00012000
regions.initrd_start:00000000
regions.initrd_size:00000000
num:0000002f
num:00000029
*kaslr_offset:07400000
Uncompressing Linux...
Ard Biesheuvel Sept. 6, 2017, 4:25 p.m. UTC | #5 
On 6 September 2017 at 17:22, Tony Lindgren <tony@atomide.com> wrote:
> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 03:41]:
>> On 5 September 2017 at 22:31, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>> > On 5 September 2017 at 22:27, Tony Lindgren <tony@atomide.com> wrote:
>> >> * Ard Biesheuvel <ard.biesheuvel@linaro.org> [170905 12:43]:
>> >>> Right. Well, I will try to reproduce with the BB white I have.
>> >>
>> >> Yeah that should be reproducable, I got it to happen on BBB here
>> >> after about 5 boots.
>> >>
>>
>> Unfortunately, my BB white boots happily every time.
>>
>> I have added DEBUG_UNCOMPRESS support to mach-omap2, and added some
>> debug output to the kaslr init routine
>
> OK. FYI, the DEBUG_UNCOMPRESS support works in Linux next as
> it's now using the generic 8250 debug_ll code.
>
>> For example,
>>
>> Starting kernel ...
>>
>> regions.image_size:00e00000
>> regions.pa_start:80000000
>> regions.pa_end:90000000
>> regions.zimage_start:80200000
>> regions.zimage_size:00430228
>> regions.dtb_start:82000000
>> regions.dtb_size:00009000
>> num:00000016
>> *kaslr_offset:04400000
>> Uncompressing Linux... done, booting the kernel.
>> [    0.000000] Booting Linux on physical CPU 0x0
>>  ....
>> [    0.000000] Virtual kernel memory layout:
>> [    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
>> [    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
>> [    0.000000]     vmalloc : 0xd0000000 - 0xff800000   ( 760 MB)
>> [    0.000000]     lowmem  : 0xc0000000 - 0xcfe00000   ( 254 MB)
>> [    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
>> [    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
>> [    0.000000]       .text : 0xc4408000 - 0xc4d00000   (9184 kB)
>> [    0.000000]       .init : 0xc5000000 - 0xc5100000   (1024 kB)
>> [    0.000000]       .data : 0xc5100000 - 0xc5149df0   ( 296 kB)
>> [    0.000000]        .bss : 0xc51cb000 - 0xc59a9a8c   (8059 kB)
>>
>> I have pushed the changes to my arm-kaslr-v3 branch. If you are
>> willing to spend more cycles on this, could you please run it with
>> DEBUG_LL enabled, and capture the output of the failed boots?
>
> Sure was not able to reproduce it so far on BBB. But here's
> failed boot output from logicpd-torpedo-37xx-devkit. Will
> try some more booting on BBB too.
>

Thanks Tony.

> 8< -------------------
> Kernel image @ 0x81000000 [ 0x000000 - 0x426810 ]
> ## Flattened Device Tree blob at 84000000
>    Booting using the fdt blob at 0x84000000
>    Loading Device Tree to 86feb000, end 86fff2d5 ... OK
>
> Starting kernel ...
>
> regions.image_size:00e00000
> regions.pa_start:80000000
> regions.pa_end:88000000
> regions.zimage_start:81000000
> regions.zimage_size:00437830
> regions.dtb_start:86feb000
> regions.dtb_size:00012000
> regions.initrd_start:00000000
> regions.initrd_size:00000000
> num:0000002f
> num:00000029
> *kaslr_offset:07400000
> Uncompressing Linux...

Is that all? Does it hang while decompressing the kernel?
Tony Lindgren Sept. 6, 2017, 4:31 p.m. UTC | #6 
* Ard Biesheuvel <ard.biesheuvel@linaro.org> [170906 09:26]:
> On 6 September 2017 at 17:22, Tony Lindgren <tony@atomide.com> wrote:
> > Sure was not able to reproduce it so far on BBB. But here's
> > failed boot output from logicpd-torpedo-37xx-devkit. Will
> > try some more booting on BBB too.
...
> > 8< -------------------
> > Kernel image @ 0x81000000 [ 0x000000 - 0x426810 ]
> > ## Flattened Device Tree blob at 84000000
> >    Booting using the fdt blob at 0x84000000
> >    Loading Device Tree to 86feb000, end 86fff2d5 ... OK
> >
> > Starting kernel ...
> >
> > regions.image_size:00e00000
> > regions.pa_start:80000000
> > regions.pa_end:88000000
> > regions.zimage_start:81000000
> > regions.zimage_size:00437830
> > regions.dtb_start:86feb000
> > regions.dtb_size:00012000
> > regions.initrd_start:00000000
> > regions.initrd_size:00000000
> > num:0000002f
> > num:00000029
> > *kaslr_offset:07400000
> > Uncompressing Linux...
> 
> Is that all? Does it hang while decompressing the kernel?

Yeah so it seems. If we had uncompress overwriting something
because of the increase in size it should happen on every
boot though, not once every five boots or so.

Regards,

TOny
diff mbox

Patch

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 1a0304dd388d..bbefd5f32ec2 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -1830,6 +1830,8 @@  config RANDOMIZE_BASE
  depends on MMU && AUTO_ZRELADDR
  depends on !XIP_KERNEL && !ZBOOT_ROM
  select RELOCATABLE
+ select ARM_MODULE_PLTS if MODULES
+ select MODULE_REL_CRCS if MODVERSIONS
  help
   Randomizes the virtual and physical address at which the kernel
   image is loaded, as a security feature that deters exploit attempts
diff --git a/arch/arm/include/asm/elf.h b/arch/arm/include/asm/elf.h
index f13ae153fb24..b56fc4dd27b6 100644
--- a/arch/arm/include/asm/elf.h
+++ b/arch/arm/include/asm/elf.h
@@ -50,6 +50,7 @@  typedef struct user_fp elf_fpregset_t;
 #define R_ARM_NONE 0
 #define R_ARM_PC24 1
 #define R_ARM_ABS32 2
+#define R_ARM_REL32 3
 #define R_ARM_CALL 28
 #define R_ARM_JUMP24 29
 #define R_ARM_TARGET1 38
diff --git a/arch/arm/kernel/module.c b/arch/arm/kernel/module.c
index 3ff571c2c71c..aa4d72837cd5 100644
--- a/arch/arm/kernel/module.c
+++ b/arch/arm/kernel/module.c
@@ -175,6 +175,10 @@ 
  *(u32 *)loc |= offset & 0x7fffffff;
  break;

+ case R_ARM_REL32:
+ *(u32 *)loc += sym->st_value - loc;
+ break;
+
  case R_ARM_MOVW_ABS_NC:
  case R_ARM_MOVT_ABS:
  offset = tmp = __mem_to_opcode_arm(*(u32 *)loc);