mbox series

[v7,00/12] implement KASLR for powerpc/fsl_booke/32

Message ID 20190920094546.44948-1-yanaijie@huawei.com (mailing list archive)
Headers show
Series implement KASLR for powerpc/fsl_booke/32 | expand

Message

Jason Yan Sept. 20, 2019, 9:45 a.m. UTC
This series implements KASLR for powerpc/fsl_booke/32, as a security
feature that deters exploit attempts relying on knowledge of the location
of kernel internals.

Since CONFIG_RELOCATABLE has already supported, what we need to do is
map or copy kernel to a proper place and relocate. Freescale Book-E
parts expect lowmem to be mapped by fixed TLB entries(TLB1). The TLB1
entries are not suitable to map the kernel directly in a randomized
region, so we chose to copy the kernel to a proper place and restart to
relocate.

Entropy is derived from the banner and timer base, which will change every
build and boot. This not so much safe so additionally the bootloader may
pass entropy via the /chosen/kaslr-seed node in device tree.

We will use the first 512M of the low memory to randomize the kernel
image. The memory will be split in 64M zones. We will use the lower 8
bit of the entropy to decide the index of the 64M zone. Then we chose a
16K aligned offset inside the 64M zone to put the kernel in.

    KERNELBASE

        |-->   64M   <--|
        |               |
        +---------------+    +----------------+---------------+
        |               |....|    |kernel|    |               |
        +---------------+    +----------------+---------------+
        |                         |
        |----->   offset    <-----|

                              kernstart_virt_addr

We also check if we will overlap with some areas like the dtb area, the
initrd area or the crashkernel area. If we cannot find a proper area,
kaslr will be disabled and boot from the original kernel.

Changes since v6:
 - Rename create_tlb_entry() to create_kaslr_tlb_entry()
 - Remove MAS2_VAL since there is no more users.
 - Move kaslr_booke.c to arch/powerpc/mm/nohash.
 - Call flush_icache_range() after copying the kernel.
 - Warning if no kaslr-seed provided by the bootloader
 - Use the right physical address when checking if the new position will overlap with other regions.
 - Do not clear bss for the second pass because some global variables will not be initialized again 
 - Use tabs instead of spaces between the mnemonic and the arguments(in fsl_booke_entry_mapping.S).

Changes since v5:
 - Rename M_IF_NEEDED to MAS2_M_IF_NEEDED
 - Define some global variable as __ro_after_init
 - Replace kimage_vaddr with kernstart_virt_addr
 - Depend on RELOCATABLE, not select it
 - Modify the comment block below the SPDX tag
 - Remove some useless headers in kaslr_booke.c and move is_second_reloc
   declarationto mmu_decl.h
 - Remove DBG() and use pr_debug() and rewrite comment above get_boot_seed().
 - Add a patch to document the KASLR implementation.
 - Split a patch from patch #10 which exports kaslr offset in VMCOREINFO ELF notes.
 - Remove extra logic around finding nokaslr string in cmdline.
 - Make regions static global and __initdata

Changes since v4:
 - Add Reviewed-by tag from Christophe
 - Remove an unnecessary cast
 - Remove unnecessary parenthesis
 - Fix checkpatch warning

Changes since v3:
 - Add Reviewed-by and Tested-by tag from Diana
 - Change the comment in fsl_booke_entry_mapping.S to be consistent
   with the new code.

Changes since v2:
 - Remove unnecessary #ifdef
 - Use SZ_64M instead of0x4000000
 - Call early_init_dt_scan_chosen() to init boot_command_line
 - Rename kaslr_second_init() to kaslr_late_init()

Changes since v1:
 - Remove some useless 'extern' keyword.
 - Replace EXPORT_SYMBOL with EXPORT_SYMBOL_GPL
 - Improve some assembly code
 - Use memzero_explicit instead of memset
 - Use boot_command_line and remove early_command_line
 - Do not print kaslr offset if kaslr is disabled

Jason Yan (12):
  powerpc: unify definition of M_IF_NEEDED
  powerpc: move memstart_addr and kernstart_addr to init-common.c
  powerpc: introduce kernstart_virt_addr to store the kernel base
  powerpc/fsl_booke/32: introduce create_kaslr_tlb_entry() helper
  powerpc/fsl_booke/32: introduce reloc_kernel_entry() helper
  powerpc/fsl_booke/32: implement KASLR infrastructure
  powerpc/fsl_booke/32: randomize the kernel image offset
  powerpc/fsl_booke/kaslr: clear the original kernel if randomized
  powerpc/fsl_booke/kaslr: support nokaslr cmdline parameter
  powerpc/fsl_booke/kaslr: dump out kernel offset information on panic
  powerpc/fsl_booke/kaslr: export offset in VMCOREINFO ELF notes
  powerpc/fsl_booke/32: Document KASLR implementation

 Documentation/powerpc/kaslr-booke32.rst       |  42 ++
 arch/powerpc/Kconfig                          |  11 +
 arch/powerpc/include/asm/nohash/mmu-book3e.h  |  11 +-
 arch/powerpc/include/asm/page.h               |   7 +
 arch/powerpc/kernel/early_32.c                |   5 +-
 arch/powerpc/kernel/exceptions-64e.S          |  12 +-
 arch/powerpc/kernel/fsl_booke_entry_mapping.S |  25 +-
 arch/powerpc/kernel/head_fsl_booke.S          |  61 ++-
 arch/powerpc/kernel/machine_kexec.c           |   1 +
 arch/powerpc/kernel/misc_64.S                 |   7 +-
 arch/powerpc/kernel/setup-common.c            |  20 +
 arch/powerpc/mm/init-common.c                 |   7 +
 arch/powerpc/mm/init_32.c                     |   5 -
 arch/powerpc/mm/init_64.c                     |   5 -
 arch/powerpc/mm/mmu_decl.h                    |  11 +
 arch/powerpc/mm/nohash/Makefile               |   1 +
 arch/powerpc/mm/nohash/fsl_booke.c            |   8 +-
 arch/powerpc/mm/nohash/kaslr_booke.c          | 401 ++++++++++++++++++
 18 files changed, 587 insertions(+), 53 deletions(-)
 create mode 100644 Documentation/powerpc/kaslr-booke32.rst
 create mode 100644 arch/powerpc/mm/nohash/kaslr_booke.c

Comments

Jason Yan Sept. 24, 2019, 5:52 a.m. UTC | #1
Hi Scott,

Can you test v7 to see if it works to load a kernel at a non-zero address?

Thanks,

On 2019/9/20 17:45, Jason Yan wrote:
> This series implements KASLR for powerpc/fsl_booke/32, as a security
> feature that deters exploit attempts relying on knowledge of the location
> of kernel internals.
> 
> Since CONFIG_RELOCATABLE has already supported, what we need to do is
> map or copy kernel to a proper place and relocate. Freescale Book-E
> parts expect lowmem to be mapped by fixed TLB entries(TLB1). The TLB1
> entries are not suitable to map the kernel directly in a randomized
> region, so we chose to copy the kernel to a proper place and restart to
> relocate.
> 
> Entropy is derived from the banner and timer base, which will change every
> build and boot. This not so much safe so additionally the bootloader may
> pass entropy via the /chosen/kaslr-seed node in device tree.
> 
> We will use the first 512M of the low memory to randomize the kernel
> image. The memory will be split in 64M zones. We will use the lower 8
> bit of the entropy to decide the index of the 64M zone. Then we chose a
> 16K aligned offset inside the 64M zone to put the kernel in.
> 
>      KERNELBASE
> 
>          |-->   64M   <--|
>          |               |
>          +---------------+    +----------------+---------------+
>          |               |....|    |kernel|    |               |
>          +---------------+    +----------------+---------------+
>          |                         |
>          |----->   offset    <-----|
> 
>                                kernstart_virt_addr
> 
> We also check if we will overlap with some areas like the dtb area, the
> initrd area or the crashkernel area. If we cannot find a proper area,
> kaslr will be disabled and boot from the original kernel.
> 
> Changes since v6:
>   - Rename create_tlb_entry() to create_kaslr_tlb_entry()
>   - Remove MAS2_VAL since there is no more users.
>   - Move kaslr_booke.c to arch/powerpc/mm/nohash.
>   - Call flush_icache_range() after copying the kernel.
>   - Warning if no kaslr-seed provided by the bootloader
>   - Use the right physical address when checking if the new position will overlap with other regions.
>   - Do not clear bss for the second pass because some global variables will not be initialized again
>   - Use tabs instead of spaces between the mnemonic and the arguments(in fsl_booke_entry_mapping.S).
> 
> Changes since v5:
>   - Rename M_IF_NEEDED to MAS2_M_IF_NEEDED
>   - Define some global variable as __ro_after_init
>   - Replace kimage_vaddr with kernstart_virt_addr
>   - Depend on RELOCATABLE, not select it
>   - Modify the comment block below the SPDX tag
>   - Remove some useless headers in kaslr_booke.c and move is_second_reloc
>     declarationto mmu_decl.h
>   - Remove DBG() and use pr_debug() and rewrite comment above get_boot_seed().
>   - Add a patch to document the KASLR implementation.
>   - Split a patch from patch #10 which exports kaslr offset in VMCOREINFO ELF notes.
>   - Remove extra logic around finding nokaslr string in cmdline.
>   - Make regions static global and __initdata
> 
> Changes since v4:
>   - Add Reviewed-by tag from Christophe
>   - Remove an unnecessary cast
>   - Remove unnecessary parenthesis
>   - Fix checkpatch warning
> 
> Changes since v3:
>   - Add Reviewed-by and Tested-by tag from Diana
>   - Change the comment in fsl_booke_entry_mapping.S to be consistent
>     with the new code.
> 
> Changes since v2:
>   - Remove unnecessary #ifdef
>   - Use SZ_64M instead of0x4000000
>   - Call early_init_dt_scan_chosen() to init boot_command_line
>   - Rename kaslr_second_init() to kaslr_late_init()
> 
> Changes since v1:
>   - Remove some useless 'extern' keyword.
>   - Replace EXPORT_SYMBOL with EXPORT_SYMBOL_GPL
>   - Improve some assembly code
>   - Use memzero_explicit instead of memset
>   - Use boot_command_line and remove early_command_line
>   - Do not print kaslr offset if kaslr is disabled
> 
> Jason Yan (12):
>    powerpc: unify definition of M_IF_NEEDED
>    powerpc: move memstart_addr and kernstart_addr to init-common.c
>    powerpc: introduce kernstart_virt_addr to store the kernel base
>    powerpc/fsl_booke/32: introduce create_kaslr_tlb_entry() helper
>    powerpc/fsl_booke/32: introduce reloc_kernel_entry() helper
>    powerpc/fsl_booke/32: implement KASLR infrastructure
>    powerpc/fsl_booke/32: randomize the kernel image offset
>    powerpc/fsl_booke/kaslr: clear the original kernel if randomized
>    powerpc/fsl_booke/kaslr: support nokaslr cmdline parameter
>    powerpc/fsl_booke/kaslr: dump out kernel offset information on panic
>    powerpc/fsl_booke/kaslr: export offset in VMCOREINFO ELF notes
>    powerpc/fsl_booke/32: Document KASLR implementation
> 
>   Documentation/powerpc/kaslr-booke32.rst       |  42 ++
>   arch/powerpc/Kconfig                          |  11 +
>   arch/powerpc/include/asm/nohash/mmu-book3e.h  |  11 +-
>   arch/powerpc/include/asm/page.h               |   7 +
>   arch/powerpc/kernel/early_32.c                |   5 +-
>   arch/powerpc/kernel/exceptions-64e.S          |  12 +-
>   arch/powerpc/kernel/fsl_booke_entry_mapping.S |  25 +-
>   arch/powerpc/kernel/head_fsl_booke.S          |  61 ++-
>   arch/powerpc/kernel/machine_kexec.c           |   1 +
>   arch/powerpc/kernel/misc_64.S                 |   7 +-
>   arch/powerpc/kernel/setup-common.c            |  20 +
>   arch/powerpc/mm/init-common.c                 |   7 +
>   arch/powerpc/mm/init_32.c                     |   5 -
>   arch/powerpc/mm/init_64.c                     |   5 -
>   arch/powerpc/mm/mmu_decl.h                    |  11 +
>   arch/powerpc/mm/nohash/Makefile               |   1 +
>   arch/powerpc/mm/nohash/fsl_booke.c            |   8 +-
>   arch/powerpc/mm/nohash/kaslr_booke.c          | 401 ++++++++++++++++++
>   18 files changed, 587 insertions(+), 53 deletions(-)
>   create mode 100644 Documentation/powerpc/kaslr-booke32.rst
>   create mode 100644 arch/powerpc/mm/nohash/kaslr_booke.c
>
Jason Yan Oct. 9, 2019, 6:10 a.m. UTC | #2
Hi Scott,

Would you please take sometime to test this?

Thank you so much.

On 2019/9/24 13:52, Jason Yan wrote:
> Hi Scott,
> 
> Can you test v7 to see if it works to load a kernel at a non-zero address?
> 
> Thanks,
> 
> On 2019/9/20 17:45, Jason Yan wrote:
>> This series implements KASLR for powerpc/fsl_booke/32, as a security
>> feature that deters exploit attempts relying on knowledge of the location
>> of kernel internals.
>>
>> Since CONFIG_RELOCATABLE has already supported, what we need to do is
>> map or copy kernel to a proper place and relocate. Freescale Book-E
>> parts expect lowmem to be mapped by fixed TLB entries(TLB1). The TLB1
>> entries are not suitable to map the kernel directly in a randomized
>> region, so we chose to copy the kernel to a proper place and restart to
>> relocate.
>>
>> Entropy is derived from the banner and timer base, which will change 
>> every
>> build and boot. This not so much safe so additionally the bootloader may
>> pass entropy via the /chosen/kaslr-seed node in device tree.
>>
>> We will use the first 512M of the low memory to randomize the kernel
>> image. The memory will be split in 64M zones. We will use the lower 8
>> bit of the entropy to decide the index of the 64M zone. Then we chose a
>> 16K aligned offset inside the 64M zone to put the kernel in.
>>
>>      KERNELBASE
>>
>>          |-->   64M   <--|
>>          |               |
>>          +---------------+    +----------------+---------------+
>>          |               |....|    |kernel|    |               |
>>          +---------------+    +----------------+---------------+
>>          |                         |
>>          |----->   offset    <-----|
>>
>>                                kernstart_virt_addr
>>
>> We also check if we will overlap with some areas like the dtb area, the
>> initrd area or the crashkernel area. If we cannot find a proper area,
>> kaslr will be disabled and boot from the original kernel.
>>
>> Changes since v6:
>>   - Rename create_tlb_entry() to create_kaslr_tlb_entry()
>>   - Remove MAS2_VAL since there is no more users.
>>   - Move kaslr_booke.c to arch/powerpc/mm/nohash.
>>   - Call flush_icache_range() after copying the kernel.
>>   - Warning if no kaslr-seed provided by the bootloader
>>   - Use the right physical address when checking if the new position 
>> will overlap with other regions.
>>   - Do not clear bss for the second pass because some global variables 
>> will not be initialized again
>>   - Use tabs instead of spaces between the mnemonic and the 
>> arguments(in fsl_booke_entry_mapping.S).
>>
>> Changes since v5:
>>   - Rename M_IF_NEEDED to MAS2_M_IF_NEEDED
>>   - Define some global variable as __ro_after_init
>>   - Replace kimage_vaddr with kernstart_virt_addr
>>   - Depend on RELOCATABLE, not select it
>>   - Modify the comment block below the SPDX tag
>>   - Remove some useless headers in kaslr_booke.c and move is_second_reloc
>>     declarationto mmu_decl.h
>>   - Remove DBG() and use pr_debug() and rewrite comment above 
>> get_boot_seed().
>>   - Add a patch to document the KASLR implementation.
>>   - Split a patch from patch #10 which exports kaslr offset in 
>> VMCOREINFO ELF notes.
>>   - Remove extra logic around finding nokaslr string in cmdline.
>>   - Make regions static global and __initdata
>>
>> Changes since v4:
>>   - Add Reviewed-by tag from Christophe
>>   - Remove an unnecessary cast
>>   - Remove unnecessary parenthesis
>>   - Fix checkpatch warning
>>
>> Changes since v3:
>>   - Add Reviewed-by and Tested-by tag from Diana
>>   - Change the comment in fsl_booke_entry_mapping.S to be consistent
>>     with the new code.
>>
>> Changes since v2:
>>   - Remove unnecessary #ifdef
>>   - Use SZ_64M instead of0x4000000
>>   - Call early_init_dt_scan_chosen() to init boot_command_line
>>   - Rename kaslr_second_init() to kaslr_late_init()
>>
>> Changes since v1:
>>   - Remove some useless 'extern' keyword.
>>   - Replace EXPORT_SYMBOL with EXPORT_SYMBOL_GPL
>>   - Improve some assembly code
>>   - Use memzero_explicit instead of memset
>>   - Use boot_command_line and remove early_command_line
>>   - Do not print kaslr offset if kaslr is disabled
>>
>> Jason Yan (12):
>>    powerpc: unify definition of M_IF_NEEDED
>>    powerpc: move memstart_addr and kernstart_addr to init-common.c
>>    powerpc: introduce kernstart_virt_addr to store the kernel base
>>    powerpc/fsl_booke/32: introduce create_kaslr_tlb_entry() helper
>>    powerpc/fsl_booke/32: introduce reloc_kernel_entry() helper
>>    powerpc/fsl_booke/32: implement KASLR infrastructure
>>    powerpc/fsl_booke/32: randomize the kernel image offset
>>    powerpc/fsl_booke/kaslr: clear the original kernel if randomized
>>    powerpc/fsl_booke/kaslr: support nokaslr cmdline parameter
>>    powerpc/fsl_booke/kaslr: dump out kernel offset information on panic
>>    powerpc/fsl_booke/kaslr: export offset in VMCOREINFO ELF notes
>>    powerpc/fsl_booke/32: Document KASLR implementation
>>
>>   Documentation/powerpc/kaslr-booke32.rst       |  42 ++
>>   arch/powerpc/Kconfig                          |  11 +
>>   arch/powerpc/include/asm/nohash/mmu-book3e.h  |  11 +-
>>   arch/powerpc/include/asm/page.h               |   7 +
>>   arch/powerpc/kernel/early_32.c                |   5 +-
>>   arch/powerpc/kernel/exceptions-64e.S          |  12 +-
>>   arch/powerpc/kernel/fsl_booke_entry_mapping.S |  25 +-
>>   arch/powerpc/kernel/head_fsl_booke.S          |  61 ++-
>>   arch/powerpc/kernel/machine_kexec.c           |   1 +
>>   arch/powerpc/kernel/misc_64.S                 |   7 +-
>>   arch/powerpc/kernel/setup-common.c            |  20 +
>>   arch/powerpc/mm/init-common.c                 |   7 +
>>   arch/powerpc/mm/init_32.c                     |   5 -
>>   arch/powerpc/mm/init_64.c                     |   5 -
>>   arch/powerpc/mm/mmu_decl.h                    |  11 +
>>   arch/powerpc/mm/nohash/Makefile               |   1 +
>>   arch/powerpc/mm/nohash/fsl_booke.c            |   8 +-
>>   arch/powerpc/mm/nohash/kaslr_booke.c          | 401 ++++++++++++++++++
>>   18 files changed, 587 insertions(+), 53 deletions(-)
>>   create mode 100644 Documentation/powerpc/kaslr-booke32.rst
>>   create mode 100644 arch/powerpc/mm/nohash/kaslr_booke.c
>>
> 
> 
> .
>
Crystal Wood Oct. 9, 2019, 7:13 a.m. UTC | #3
On Wed, 2019-10-09 at 14:10 +0800, Jason Yan wrote:
> Hi Scott,
> 
> Would you please take sometime to test this?
> 
> Thank you so much.
> 
> On 2019/9/24 13:52, Jason Yan wrote:
> > Hi Scott,
> > 
> > Can you test v7 to see if it works to load a kernel at a non-zero address?
> > 
> > Thanks,

Sorry for the delay.  Here's the output:

## Booting kernel from Legacy Image at 10000000 ...
   Image Name:   Linux-5.4.0-rc2-00050-g8ac2cf5b4
   Image Type:   PowerPC Linux Kernel Image (gzip compressed)
   Data Size:    7521134 Bytes = 7.2 MiB
   Load Address: 04000000
   Entry Point:  04000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 1fc00000
   Booting using the fdt blob at 0x1fc00000
   Uncompressing Kernel Image ... OK
   Loading Device Tree to 07fe0000, end 07fff65c ... OK
KASLR: No safe seed for randomizing the kernel base.
OF: reserved mem: initialized node qman-fqd, compatible id fsl,qman-fqd
OF: reserved mem: initialized node qman-pfdr, compatible id fsl,qman-pfdr
OF: reserved mem: initialized node bman-fbpr, compatible id fsl,bman-fbpr
Memory CAM mapping: 64/64/64 Mb, residual: 12032Mb
Linux version 5.4.0-rc2-00050-g8ac2cf5b4e4a-dirty (scott@snotra) (gcc version 8.
1.0 (GCC)) #26 SMP Wed Oct 9 01:50:40 CDT 2019
Using CoreNet Generic machine description
printk: bootconsole [udbg0] enabled
CPU maps initialized for 1 thread per core
-----------------------------------------------------
phys_mem_size     = 0x2fc000000
dcache_bsize      = 0x40
icache_bsize      = 0x40
cpu_features      = 0x00000000000003b4
  possible        = 0x00000000010103bc
  always          = 0x0000000000000020
cpu_user_features = 0x8c008000 0x08000000
mmu_features      = 0x000a0010
physical_start    = 0xc7c4000
-----------------------------------------------------
CoreNet Generic board
mpc85xx_qe_init: Could not find Quicc Engine node
barrier-nospec: using isync; sync as speculation barrier
Zone ranges:
  Normal   [mem 0x0000000004000000-0x000000000fffffff]
  HighMem  [mem 0x0000000010000000-0x00000002ffffffff]
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000004000000-0x00000002ffffffff]
Initmem setup node 0 [mem 0x0000000004000000-0x00000002ffffffff]
Kernel panic - not syncing: Failed to allocate 125173760 bytes for node 0 memory
 map
CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.0-rc2-00050-g8ac2cf5b4e4a-dirty #26
Call Trace:
[c989fe10] [c924bfb0] dump_stack+0x84/0xb4 (unreliable)
[c989fe30] [c880badc] panic+0x140/0x334
[c989fe90] [c89a1144] alloc_node_mem_map.constprop.117+0xa0/0x11c
[c989feb0] [c95481c4] free_area_init_node+0x314/0x5b8
[c989ff30] [c9548b34] free_area_init_nodes+0x57c/0x5c0
[c989ff80] [c952cbb4] setup_arch+0x250/0x270
[c989ffa0] [c95278e0] start_kernel+0x74/0x4e8
[c989fff0] [c87c4478] set_ivor+0x150/0x18c
Kernel Offset: 0x87c4000 from 0xc0000000
Rebooting in 180 seconds..

-Scott
Jason Yan Oct. 9, 2019, 8:41 a.m. UTC | #4
Hi Scott,

On 2019/10/9 15:13, Scott Wood wrote:
> On Wed, 2019-10-09 at 14:10 +0800, Jason Yan wrote:
>> Hi Scott,
>>
>> Would you please take sometime to test this?
>>
>> Thank you so much.
>>
>> On 2019/9/24 13:52, Jason Yan wrote:
>>> Hi Scott,
>>>
>>> Can you test v7 to see if it works to load a kernel at a non-zero address?
>>>
>>> Thanks,
> 
> Sorry for the delay.  Here's the output:
> 

Thanks for the test.

> ## Booting kernel from Legacy Image at 10000000 ...
>     Image Name:   Linux-5.4.0-rc2-00050-g8ac2cf5b4
>     Image Type:   PowerPC Linux Kernel Image (gzip compressed)
>     Data Size:    7521134 Bytes = 7.2 MiB
>     Load Address: 04000000
>     Entry Point:  04000000
>     Verifying Checksum ... OK
> ## Flattened Device Tree blob at 1fc00000
>     Booting using the fdt blob at 0x1fc00000
>     Uncompressing Kernel Image ... OK
>     Loading Device Tree to 07fe0000, end 07fff65c ... OK
> KASLR: No safe seed for randomizing the kernel base.
> OF: reserved mem: initialized node qman-fqd, compatible id fsl,qman-fqd
> OF: reserved mem: initialized node qman-pfdr, compatible id fsl,qman-pfdr
> OF: reserved mem: initialized node bman-fbpr, compatible id fsl,bman-fbpr
> Memory CAM mapping: 64/64/64 Mb, residual: 12032Mb

When boot from 04000000, the max CAM value is 64M. And
you have a board with 12G memory, CONFIG_LOWMEM_CAM_NUM=3 means only
192M memory is mapped and when kernel is randomized at the middle of 
this 192M memory, we will not have enough continuous memory for node map.

Can you set CONFIG_LOWMEM_CAM_NUM=8 and see if it works?

Thanks.

> Linux version 5.4.0-rc2-00050-g8ac2cf5b4e4a-dirty (scott@snotra) (gcc version 8.
> 1.0 (GCC)) #26 SMP Wed Oct 9 01:50:40 CDT 2019
> Using CoreNet Generic machine description
> printk: bootconsole [udbg0] enabled
> CPU maps initialized for 1 thread per core
> -----------------------------------------------------
> phys_mem_size     = 0x2fc000000
> dcache_bsize      = 0x40
> icache_bsize      = 0x40
> cpu_features      = 0x00000000000003b4
>    possible        = 0x00000000010103bc
>    always          = 0x0000000000000020
> cpu_user_features = 0x8c008000 0x08000000
> mmu_features      = 0x000a0010
> physical_start    = 0xc7c4000
> -----------------------------------------------------
> CoreNet Generic board
> mpc85xx_qe_init: Could not find Quicc Engine node
> barrier-nospec: using isync; sync as speculation barrier
> Zone ranges:
>    Normal   [mem 0x0000000004000000-0x000000000fffffff]
>    HighMem  [mem 0x0000000010000000-0x00000002ffffffff]
> Movable zone start for each node
> Early memory node ranges
>    node   0: [mem 0x0000000004000000-0x00000002ffffffff]
> Initmem setup node 0 [mem 0x0000000004000000-0x00000002ffffffff]
> Kernel panic - not syncing: Failed to allocate 125173760 bytes for node 0 memory
>   map
> CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.0-rc2-00050-g8ac2cf5b4e4a-dirty #26
> Call Trace:
> [c989fe10] [c924bfb0] dump_stack+0x84/0xb4 (unreliable)
> [c989fe30] [c880badc] panic+0x140/0x334
> [c989fe90] [c89a1144] alloc_node_mem_map.constprop.117+0xa0/0x11c
> [c989feb0] [c95481c4] free_area_init_node+0x314/0x5b8
> [c989ff30] [c9548b34] free_area_init_nodes+0x57c/0x5c0
> [c989ff80] [c952cbb4] setup_arch+0x250/0x270
> [c989ffa0] [c95278e0] start_kernel+0x74/0x4e8
> [c989fff0] [c87c4478] set_ivor+0x150/0x18c
> Kernel Offset: 0x87c4000 from 0xc0000000
> Rebooting in 180 seconds..
> 
> -Scott
> 
> 
> 
> .
>
Crystal Wood Oct. 9, 2019, 6:46 p.m. UTC | #5
On Wed, 2019-10-09 at 16:41 +0800, Jason Yan wrote:
> Hi Scott,
> 
> On 2019/10/9 15:13, Scott Wood wrote:
> > On Wed, 2019-10-09 at 14:10 +0800, Jason Yan wrote:
> > > Hi Scott,
> > > 
> > > Would you please take sometime to test this?
> > > 
> > > Thank you so much.
> > > 
> > > On 2019/9/24 13:52, Jason Yan wrote:
> > > > Hi Scott,
> > > > 
> > > > Can you test v7 to see if it works to load a kernel at a non-zero
> > > > address?
> > > > 
> > > > Thanks,
> > 
> > Sorry for the delay.  Here's the output:
> > 
> 
> Thanks for the test.
> 
> > ## Booting kernel from Legacy Image at 10000000 ...
> >     Image Name:   Linux-5.4.0-rc2-00050-g8ac2cf5b4
> >     Image Type:   PowerPC Linux Kernel Image (gzip compressed)
> >     Data Size:    7521134 Bytes = 7.2 MiB
> >     Load Address: 04000000
> >     Entry Point:  04000000
> >     Verifying Checksum ... OK
> > ## Flattened Device Tree blob at 1fc00000
> >     Booting using the fdt blob at 0x1fc00000
> >     Uncompressing Kernel Image ... OK
> >     Loading Device Tree to 07fe0000, end 07fff65c ... OK
> > KASLR: No safe seed for randomizing the kernel base.
> > OF: reserved mem: initialized node qman-fqd, compatible id fsl,qman-fqd
> > OF: reserved mem: initialized node qman-pfdr, compatible id fsl,qman-pfdr
> > OF: reserved mem: initialized node bman-fbpr, compatible id fsl,bman-fbpr
> > Memory CAM mapping: 64/64/64 Mb, residual: 12032Mb
> 
> When boot from 04000000, the max CAM value is 64M. And
> you have a board with 12G memory, CONFIG_LOWMEM_CAM_NUM=3 means only
> 192M memory is mapped and when kernel is randomized at the middle of 
> this 192M memory, we will not have enough continuous memory for node map.
> 
> Can you set CONFIG_LOWMEM_CAM_NUM=8 and see if it works?

OK, that worked.

-Scott
Jason Yan Oct. 21, 2019, 3:34 a.m. UTC | #6
On 2019/10/10 2:46, Scott Wood wrote:
> On Wed, 2019-10-09 at 16:41 +0800, Jason Yan wrote:
>> Hi Scott,
>>
>> On 2019/10/9 15:13, Scott Wood wrote:
>>> On Wed, 2019-10-09 at 14:10 +0800, Jason Yan wrote:
>>>> Hi Scott,
>>>>
>>>> Would you please take sometime to test this?
>>>>
>>>> Thank you so much.
>>>>
>>>> On 2019/9/24 13:52, Jason Yan wrote:
>>>>> Hi Scott,
>>>>>
>>>>> Can you test v7 to see if it works to load a kernel at a non-zero
>>>>> address?
>>>>>
>>>>> Thanks,
>>>
>>> Sorry for the delay.  Here's the output:
>>>
>>
>> Thanks for the test.
>>
>>> ## Booting kernel from Legacy Image at 10000000 ...
>>>      Image Name:   Linux-5.4.0-rc2-00050-g8ac2cf5b4
>>>      Image Type:   PowerPC Linux Kernel Image (gzip compressed)
>>>      Data Size:    7521134 Bytes = 7.2 MiB
>>>      Load Address: 04000000
>>>      Entry Point:  04000000
>>>      Verifying Checksum ... OK
>>> ## Flattened Device Tree blob at 1fc00000
>>>      Booting using the fdt blob at 0x1fc00000
>>>      Uncompressing Kernel Image ... OK
>>>      Loading Device Tree to 07fe0000, end 07fff65c ... OK
>>> KASLR: No safe seed for randomizing the kernel base.
>>> OF: reserved mem: initialized node qman-fqd, compatible id fsl,qman-fqd
>>> OF: reserved mem: initialized node qman-pfdr, compatible id fsl,qman-pfdr
>>> OF: reserved mem: initialized node bman-fbpr, compatible id fsl,bman-fbpr
>>> Memory CAM mapping: 64/64/64 Mb, residual: 12032Mb
>>
>> When boot from 04000000, the max CAM value is 64M. And
>> you have a board with 12G memory, CONFIG_LOWMEM_CAM_NUM=3 means only
>> 192M memory is mapped and when kernel is randomized at the middle of
>> this 192M memory, we will not have enough continuous memory for node map.
>>
>> Can you set CONFIG_LOWMEM_CAM_NUM=8 and see if it works?
> 
> OK, that worked.
> 

Hi Scott, any more cases should be tested or any more comments?
What else need to be done before this feature can be merged?

Thanks,
Jason

> -Scott
> 
> 
> 
> .
>
Crystal Wood Oct. 22, 2019, 11:22 p.m. UTC | #7
On Mon, 2019-10-21 at 11:34 +0800, Jason Yan wrote:
> 
> On 2019/10/10 2:46, Scott Wood wrote:
> > On Wed, 2019-10-09 at 16:41 +0800, Jason Yan wrote:
> > > Hi Scott,
> > > 
> > > On 2019/10/9 15:13, Scott Wood wrote:
> > > > On Wed, 2019-10-09 at 14:10 +0800, Jason Yan wrote:
> > > > > Hi Scott,
> > > > > 
> > > > > Would you please take sometime to test this?
> > > > > 
> > > > > Thank you so much.
> > > > > 
> > > > > On 2019/9/24 13:52, Jason Yan wrote:
> > > > > > Hi Scott,
> > > > > > 
> > > > > > Can you test v7 to see if it works to load a kernel at a non-zero
> > > > > > address?
> > > > > > 
> > > > > > Thanks,
> > > > 
> > > > Sorry for the delay.  Here's the output:
> > > > 
> > > 
> > > Thanks for the test.
> > > 
> > > > ## Booting kernel from Legacy Image at 10000000 ...
> > > >      Image Name:   Linux-5.4.0-rc2-00050-g8ac2cf5b4
> > > >      Image Type:   PowerPC Linux Kernel Image (gzip compressed)
> > > >      Data Size:    7521134 Bytes = 7.2 MiB
> > > >      Load Address: 04000000
> > > >      Entry Point:  04000000
> > > >      Verifying Checksum ... OK
> > > > ## Flattened Device Tree blob at 1fc00000
> > > >      Booting using the fdt blob at 0x1fc00000
> > > >      Uncompressing Kernel Image ... OK
> > > >      Loading Device Tree to 07fe0000, end 07fff65c ... OK
> > > > KASLR: No safe seed for randomizing the kernel base.
> > > > OF: reserved mem: initialized node qman-fqd, compatible id fsl,qman-
> > > > fqd
> > > > OF: reserved mem: initialized node qman-pfdr, compatible id fsl,qman-
> > > > pfdr
> > > > OF: reserved mem: initialized node bman-fbpr, compatible id fsl,bman-
> > > > fbpr
> > > > Memory CAM mapping: 64/64/64 Mb, residual: 12032Mb
> > > 
> > > When boot from 04000000, the max CAM value is 64M. And
> > > you have a board with 12G memory, CONFIG_LOWMEM_CAM_NUM=3 means only
> > > 192M memory is mapped and when kernel is randomized at the middle of
> > > this 192M memory, we will not have enough continuous memory for node
> > > map.
> > > 
> > > Can you set CONFIG_LOWMEM_CAM_NUM=8 and see if it works?
> > 
> > OK, that worked.
> > 
> 
> Hi Scott, any more cases should be tested or any more comments?
> What else need to be done before this feature can be merged?

I've just applied it and sent a pull request.

-Scott