[00/10] Kendryte k210 SoC boards support

Message ID	20200212103432.660256-1-damien.lemoal@wdc.com (mailing list archive)
Headers	show Return-Path: <SRS0=ZxAA=4A=lists.infradead.org=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@kernel.org> DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 554A32073C IronPort-SDR: g0n04Y5I06TvY3nN8YeQrXNi/XEKOTPq/wNF/Wk5xUV0GlG4tsLHPAqlb2yG9k+8OH9rW8JyWq H3rF82MGR4DRWXAOaQGJ5E3ZZ4FIIhMGXBWuDGnyD4IxLDxixaoVIuMPycxEDmx4n35h+oNAXn RLQOdsESa8OKaWOXKeLrIgnlG9lxfIXktGVa6YhCMmt5K4UAD2o0PaO4ZwA046JRiB1Dd9c1q0 jihK+gqwGq1hs3WHybZw2hgPXKGsvqHv8A30bfYh6stHi6ySj0/RS6TqdcdCEj6j6593vU0Oc3 ls0= IronPort-SDR: znx8cK1Qj2mLM/VSITTsIiE3Sw5JjqNbBnhrxEh46alHOyQK32GxfxlIWEb609GrVoHUDnAZSF 8ELeBlM5Q7wnkfpPuNN6dAzXevZ57hHx5985/ll5xFdj7HN8zZG7Dny4Z/vKYFVpU7vRggzRC7 VbBiJ2BeIQH85Bpk1K2Ne9KmihKXeg0bMl9T6kc7PtSdxpG3lE9F0KSRBvBWEGtsm9Z2fq9zPA on/xEkIwCiCIWduRROfgzncImR4FphO5el5C6hU1CQGu5J6cARFaAKvjhy0wmmqvm1EpO2S03g thtJLa8UL3lVIXUAWbc0xTjK IronPort-SDR: UYR5J+b82oRTRsP6bBgY/8XCgOtV4LIVcnlto9fThXviPT7PSFZyGaPKMuvmyuNsp6Kpb5yBDP h2eHu2BrvfIaaBDWN8wkKdww1XKyBapfN/WtcZzK+dh1t0ctP11QMgBiDlDmenBthj+iYyHDCR iEOETn84rdRU57KZ0F+BFMldWpJ8ZmwZ2rm6DkrLLvmLHiEg+APP6ruPbXLA2Mx1fWeNAgbrwl Rk5+Si5RHtGzzszJiBU0IOjLWjzcZ5d+31jHXZGAD07oCI+5YsX19zSI1BBCwHySe4DLhQ70MJ z0I= WDCIronportException: Internal From: Damien Le Moal <damien.lemoal@wdc.com> To: linux-riscv@lists.infradead.org, Palmer Dabbelt <palmer@dabbelt.com> Subject: [PATCH 00/10] Kendryte k210 SoC boards support Date: Wed, 12 Feb 2020 19:34:22 +0900 Message-Id: <20200212103432.660256-1-damien.lemoal@wdc.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit summary: Content analysis details: (-2.5 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [216.71.154.45 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain Precedence: list Cc: Anup Patel <Anup.Patel@wdc.com>, Paul Walmsley <paul.walmsley@sifive.com> Sender: "linux-riscv" <linux-riscv-bounces@lists.infradead.org> Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org
Series	Kendryte k210 SoC boards support \| expand [00/10] Kendryte k210 SoC boards support [01/10] riscv: Fix gitignore [02/10] riscv: Force flat memory model with no-mmu [03/10] riscv: Unaligned load/store handling for M_MODE [04/10] riscv: Add BUILTIN_DTB support [05/10] riscv: Add SOC early init support [06/10] riscv: Add Kendryte K210 SoC support [07/10] riscv: Select required drivers for Kendryte SOC [08/10] riscv: Add Kendryte K210 device tree [09/10] riscv: Kendryte K210 default config [10/10] riscv: create a loader.bin for the kendryte kflash.py tool

Damien Le Moal Feb. 12, 2020, 10:34 a.m. UTC

This series adds support to boot nommu Linux on Kendryte K210 SoC based
boards. This is all based on initial work done by Christoph Hellwig.

The first 2 patches fix riscv gitignore and a potential nommu
compilation error. These patches are not specific to the Kendryte
support.

Patch 3 adds unaligned load/store trap handlers for M-mode.

Patch 4 enables a builtin DTB to allow passing a device tree to the
kernel when the board bootchain is enabled to pass one.

Patch 5 introduces an early SoC initialization enabling very early
hardware initialization not possible with device tree entries pointing
to drivers. This is used in patch 6 which introduces a sysctl driver for
the K210 SoC. The early SoC initialization is used to enable the
additional 2MB of SRAM normally reserved to the SoC AI chip.

Patch 7 to 9 add necessary Kconfig changes, a defconfig and a generic
device tree suitable for many K210 boards.

Finally, patch 10 adds compilation of a bootable image file (bin file)
that can be used to flash the board ROM.

This series was tested on the Kendryte KD233 development board, the
Sipeed MAIX dan dock board and the Sipeed MAIXDUINO board. The userspace
used was built using a modified buildroot tree for the toolchain part
and an unmodified busybox tree for the initramfs image (embedded in the
kernel as a cpio file). The folowwing github project contains the
modified buildroot tree:

https://github.com/damien-lemoal/riscv64-nommu-buildroot

This is based on work from Christoph Hellwig, with additional changes
and updates to the latest upstream versions for buildroot and uClibc.

Precompiled versions of the toolchain (gcc 9.2) and initramfs file tree
and cpio file can be found in this project under the directory:

buildroot/riscv64-uclibc-nommu/

Flashing the file arch/riscv/boot/loader.bin to a board can be done
using the Sipeed kflash.py tool with the command:

kflash.py/kflash.py -p /dev/ttyUSB0 -b 1500000 -t loader.bin

The kflash.py tool can be found here:

https://github.com/sipeed/kflash.py

For reference, using the Sipeed MAIXDUINO board, here is the boot
output:

[INFO] Rebooting... 
--- forcing DTR inactive
--- forcing RTS inactive
--- Miniterm on /dev/ttyUSB0  115200,8,N,1 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
[    0.000000] Linux version 5.6.0-rc1-00011-ga2b5be1c4374 (damien@yyy) (gcc version 8.2.0 (Buildroot 2018.11-rc2-00003-ga0787e9)) #610 SMP Wed Feb 12 18:53:50 JST 2020
[    0.000000] earlycon: sifive0 at MMIO 0x0000000038000000 (options '')
[    0.000000] printk: bootconsole [sifive0] enabled
[    0.000000] initrd not found or empty - disabling initrd
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000080000000-0x00000000807fffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080000000-0x00000000807fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x00000000807fffff]
[    0.000000] elf_hwcap is 0x112d
[    0.000000] percpu: max_distance=0x18000 too large for vmalloc space 0x0
[    0.000000] percpu: Embedded 12 pages/cpu s18272 r0 d30880 u49152
[    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2020
[    0.000000] Kernel command line: earlycon console=ttySIF0
[    0.000000] Dentry cache hash table entries: 1024 (order: 1, 8192 bytes, linear)
[    0.000000] Inode-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
[    0.000000] Sorting __ex_table...
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 6328K/8192K available (924K kernel code, 110K rwdata, 164K rodata, 321K init, 91K bss, 1864K reserved, 0K cma-reserved)
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] NR_IRQS: 0, nr_irqs: 0, preallocated irqs: 0
[    0.000000] plic: mapped 65 interrupts with 2 handlers for 4 contexts.
[    0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x3990be68b, max_idle_ns: 881590404272 ns
[    0.000014] sched_clock: 64 bits at 7MHz, resolution 128ns, wraps every 4398046511054ns
[    0.008232] Console: colour dummy device 80x25
[    0.012474] Calibrating delay loop (skipped), value calculated using timer frequency.. 15.60 BogoMIPS (lpj=31200)
[    0.022678] pid_max: default: 4096 minimum: 301
[    0.027288] Mount-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
[    0.034414] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
[    0.044796] rcu: Hierarchical SRCU implementation.
[    0.049602] smp: Bringing up secondary CPUs ...
[    0.054746] smp: Brought up 1 node, 2 CPUs
[    0.059093] devtmpfs: initialized
[    0.065523] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.074544] futex hash table entries: 16 (order: -2, 1024 bytes, linear)
[    0.082512] Kendryte K210 SoC sysctl
[    0.096010] clocksource: Switched to clocksource riscv_clocksource
[    0.178581] workingset: timestamp_bits=62 max_order=11 bucket_order=0
[    0.185846] 38000000.serial: ttySIF0 at MMIO 0x38000000 (irq = 1, base_baud = 0) is a SiFive UART v0
[    0.194344] printk: console [ttySIF0] enabled
[    0.194344] printk: console [ttySIF0] enabled
[    0.202929] printk: bootconsole [sifive0] disabled
[    0.202929] printk: bootconsole [sifive0] disabled
[    0.214853] random: get_random_bytes called from 0x0000000080055178 with crng_init=0
[    0.223606] devtmpfs: mounted
[    0.226861] Freeing unused kernel memory: 320K
[    0.230574] This architecture does not have kernel memory protection.
[    0.236987] Run /sbin/init as init process
[    0.241181] Run /etc/init as init process
[    0.245178] Run /bin/init as init process

-----------------------------
| Kendryte K210 NOMMU Linux |
-----------------------------
Mounting /proc
Starting shell


BusyBox v1.32.0.git (2020-02-12 17:51:45 JST) hush - the humble shell
Enter 'help' for a list of built-in commands.

/ # cat /proc/cpuinfo 
processor	: 0
hart		: 0
isa		: rv64imafdc

processor	: 1
hart		: 1
isa		: rv64imafdc

/ # 
/ # ls -l /
drwxrwxr-x    2 1000     1000             0 Feb 12  2020 bin
drwxr-xr-x    2 0        0                0 Jan  1 00:00 dev
drwxrwxr-x    2 1000     1000             0 Feb 12  2020 etc
dr-xr-xr-x   58 0        0                0 Jan  1 00:00 proc
drwxrwxr-x    2 1000     1000             0 Feb 12  2020 root
drwxrwxr-x    2 1000     1000             0 Feb 12  2020 sbin
drwxrwxr-x    2 1000     1000             0 Feb 12  2020 sys
drwxrwxr-x    2 1000     1000             0 Feb 12  2020 tmp
drwxrwxr-x    4 1000     1000             0 Feb 12  2020 usr
/ # 
/ # cat /proc/vmstat 
nr_free_pages 1148
...
/ #

The K210 SoC has more devices (GPIO, SD-card, LCD, etc) that likely can
be enabled and used. For this, the sysctl driver will need further
improvements as each device uses a different clock. To share the fun,
supporting these is left as an exercise for the hobbyist and hackers
interested in this SoC/boards :)

Christoph Hellwig (2):
  riscv: Add Kendryte K210 SoC support
  riscv: create a loader.bin for the kendryte kflash.py tool

Damien Le Moal (8):
  riscv: Fix gitignore
  riscv: Force flat memory model with no-mmu
  riscv: Unaligned load/store handling for M_MODE
  riscv: Add BUILTIN_DTB support
  riscv: Add SOC early init support
  riscv: Select required drivers for Kendryte SOC
  riscv: Add Kendryte K210 device tree
  riscv: Kendryte K210 default config

 arch/riscv/Kbuild                       |   1 +
 arch/riscv/Kconfig                      |  19 ++
 arch/riscv/Kconfig.socs                 |  10 +
 arch/riscv/Makefile                     |   4 +-
 arch/riscv/boot/.gitignore              |   2 +
 arch/riscv/boot/Makefile                |   3 +
 arch/riscv/boot/dts/Makefile            |   5 +
 arch/riscv/boot/dts/kendryte/Makefile   |   2 +
 arch/riscv/boot/dts/kendryte/k210.dts   |  23 ++
 arch/riscv/boot/dts/kendryte/k210.dtsi  | 123 ++++++++
 arch/riscv/configs/nommu_k210_defconfig |  68 +++++
 arch/riscv/include/asm/soc.h            |  23 ++
 arch/riscv/kernel/Makefile              |   3 +-
 arch/riscv/kernel/head.S                |   1 +
 arch/riscv/kernel/setup.c               |   6 +
 arch/riscv/kernel/soc.c                 |  28 ++
 arch/riscv/kernel/traps.c               |  27 +-
 arch/riscv/kernel/traps_misaligned.c    | 371 ++++++++++++++++++++++++
 arch/riscv/kernel/vmlinux.lds.S         |   6 +
 arch/riscv/mm/init.c                    |   4 +
 drivers/soc/Kconfig                     |   1 +
 drivers/soc/Makefile                    |   1 +
 drivers/soc/kendryte/Kconfig            |  14 +
 drivers/soc/kendryte/Makefile           |   3 +
 drivers/soc/kendryte/k210-sysctl.c      | 245 ++++++++++++++++
 25 files changed, 987 insertions(+), 6 deletions(-)
 create mode 100644 arch/riscv/boot/dts/kendryte/Makefile
 create mode 100644 arch/riscv/boot/dts/kendryte/k210.dts
 create mode 100644 arch/riscv/boot/dts/kendryte/k210.dtsi
 create mode 100644 arch/riscv/configs/nommu_k210_defconfig
 create mode 100644 arch/riscv/include/asm/soc.h
 create mode 100644 arch/riscv/kernel/soc.c
 create mode 100644 arch/riscv/kernel/traps_misaligned.c
 create mode 100644 drivers/soc/kendryte/Kconfig
 create mode 100644 drivers/soc/kendryte/Makefile
 create mode 100644 drivers/soc/kendryte/k210-sysctl.c

Carlos de Paula Feb. 14, 2020, 3:05 p.m. UTC | #1

On Wed, Feb 12, 2020 at 7:34 AM Damien Le Moal <damien.lemoal@wdc.com> wrote:
>
> This series adds support to boot nommu Linux on Kendryte K210 SoC based
> boards. This is all based on initial work done by Christoph Hellwig.
>
> The first 2 patches fix riscv gitignore and a potential nommu
> compilation error. These patches are not specific to the Kendryte
> support.
>
> Patch 3 adds unaligned load/store trap handlers for M-mode.
>
> Patch 4 enables a builtin DTB to allow passing a device tree to the
> kernel when the board bootchain is enabled to pass one.
>
> Patch 5 introduces an early SoC initialization enabling very early
> hardware initialization not possible with device tree entries pointing
> to drivers. This is used in patch 6 which introduces a sysctl driver for
> the K210 SoC. The early SoC initialization is used to enable the
> additional 2MB of SRAM normally reserved to the SoC AI chip.
>
> Patch 7 to 9 add necessary Kconfig changes, a defconfig and a generic
> device tree suitable for many K210 boards.
>
> Finally, patch 10 adds compilation of a bootable image file (bin file)
> that can be used to flash the board ROM.
>
> This series was tested on the Kendryte KD233 development board, the
> Sipeed MAIX dan dock board and the Sipeed MAIXDUINO board. The userspace
> used was built using a modified buildroot tree for the toolchain part
> and an unmodified busybox tree for the initramfs image (embedded in the
> kernel as a cpio file). The folowwing github project contains the
> modified buildroot tree:
>
> https://github.com/damien-lemoal/riscv64-nommu-buildroot
>
> This is based on work from Christoph Hellwig, with additional changes
> and updates to the latest upstream versions for buildroot and uClibc.
>
> Precompiled versions of the toolchain (gcc 9.2) and initramfs file tree
> and cpio file can be found in this project under the directory:
>
> buildroot/riscv64-uclibc-nommu/
>
> Flashing the file arch/riscv/boot/loader.bin to a board can be done
> using the Sipeed kflash.py tool with the command:
>
> kflash.py/kflash.py -p /dev/ttyUSB0 -b 1500000 -t loader.bin
>
> The kflash.py tool can be found here:
>
> https://github.com/sipeed/kflash.py
>
> For reference, using the Sipeed MAIXDUINO board, here is the boot
> output:
>
> [INFO] Rebooting...
> --- forcing DTR inactive
> --- forcing RTS inactive
> --- Miniterm on /dev/ttyUSB0  115200,8,N,1 ---
> --- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
> [    0.000000] Linux version 5.6.0-rc1-00011-ga2b5be1c4374 (damien@yyy) (gcc version 8.2.0 (Buildroot 2018.11-rc2-00003-ga0787e9)) #610 SMP Wed Feb 12 18:53:50 JST 2020
> [    0.000000] earlycon: sifive0 at MMIO 0x0000000038000000 (options '')
> [    0.000000] printk: bootconsole [sifive0] enabled
> [    0.000000] initrd not found or empty - disabling initrd
> [    0.000000] Zone ranges:
> [    0.000000]   DMA32    [mem 0x0000000080000000-0x00000000807fffff]
> [    0.000000]   Normal   empty
> [    0.000000] Movable zone start for each node
> [    0.000000] Early memory node ranges
> [    0.000000]   node   0: [mem 0x0000000080000000-0x00000000807fffff]
> [    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x00000000807fffff]
> [    0.000000] elf_hwcap is 0x112d
> [    0.000000] percpu: max_distance=0x18000 too large for vmalloc space 0x0
> [    0.000000] percpu: Embedded 12 pages/cpu s18272 r0 d30880 u49152
> [    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2020
> [    0.000000] Kernel command line: earlycon console=ttySIF0
> [    0.000000] Dentry cache hash table entries: 1024 (order: 1, 8192 bytes, linear)
> [    0.000000] Inode-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
> [    0.000000] Sorting __ex_table...
> [    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
> [    0.000000] Memory: 6328K/8192K available (924K kernel code, 110K rwdata, 164K rodata, 321K init, 91K bss, 1864K reserved, 0K cma-reserved)
> [    0.000000] rcu: Hierarchical RCU implementation.
> [    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
> [    0.000000] NR_IRQS: 0, nr_irqs: 0, preallocated irqs: 0
> [    0.000000] plic: mapped 65 interrupts with 2 handlers for 4 contexts.
> [    0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
> [    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x3990be68b, max_idle_ns: 881590404272 ns
> [    0.000014] sched_clock: 64 bits at 7MHz, resolution 128ns, wraps every 4398046511054ns
> [    0.008232] Console: colour dummy device 80x25
> [    0.012474] Calibrating delay loop (skipped), value calculated using timer frequency.. 15.60 BogoMIPS (lpj=31200)
> [    0.022678] pid_max: default: 4096 minimum: 301
> [    0.027288] Mount-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
> [    0.034414] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
> [    0.044796] rcu: Hierarchical SRCU implementation.
> [    0.049602] smp: Bringing up secondary CPUs ...
> [    0.054746] smp: Brought up 1 node, 2 CPUs
> [    0.059093] devtmpfs: initialized
> [    0.065523] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
> [    0.074544] futex hash table entries: 16 (order: -2, 1024 bytes, linear)
> [    0.082512] Kendryte K210 SoC sysctl
> [    0.096010] clocksource: Switched to clocksource riscv_clocksource
> [    0.178581] workingset: timestamp_bits=62 max_order=11 bucket_order=0
> [    0.185846] 38000000.serial: ttySIF0 at MMIO 0x38000000 (irq = 1, base_baud = 0) is a SiFive UART v0
> [    0.194344] printk: console [ttySIF0] enabled
> [    0.194344] printk: console [ttySIF0] enabled
> [    0.202929] printk: bootconsole [sifive0] disabled
> [    0.202929] printk: bootconsole [sifive0] disabled
> [    0.214853] random: get_random_bytes called from 0x0000000080055178 with crng_init=0
> [    0.223606] devtmpfs: mounted
> [    0.226861] Freeing unused kernel memory: 320K
> [    0.230574] This architecture does not have kernel memory protection.
> [    0.236987] Run /sbin/init as init process
> [    0.241181] Run /etc/init as init process
> [    0.245178] Run /bin/init as init process
>
> -----------------------------
> | Kendryte K210 NOMMU Linux |
> -----------------------------
> Mounting /proc
> Starting shell
>
>
> BusyBox v1.32.0.git (2020-02-12 17:51:45 JST) hush - the humble shell
> Enter 'help' for a list of built-in commands.
>
> / # cat /proc/cpuinfo
> processor       : 0
> hart            : 0
> isa             : rv64imafdc
>
> processor       : 1
> hart            : 1
> isa             : rv64imafdc
>
> / #
> / # ls -l /
> drwxrwxr-x    2 1000     1000             0 Feb 12  2020 bin
> drwxr-xr-x    2 0        0                0 Jan  1 00:00 dev
> drwxrwxr-x    2 1000     1000             0 Feb 12  2020 etc
> dr-xr-xr-x   58 0        0                0 Jan  1 00:00 proc
> drwxrwxr-x    2 1000     1000             0 Feb 12  2020 root
> drwxrwxr-x    2 1000     1000             0 Feb 12  2020 sbin
> drwxrwxr-x    2 1000     1000             0 Feb 12  2020 sys
> drwxrwxr-x    2 1000     1000             0 Feb 12  2020 tmp
> drwxrwxr-x    4 1000     1000             0 Feb 12  2020 usr
> / #
> / # cat /proc/vmstat
> nr_free_pages 1148
> ...
> / #
>
> The K210 SoC has more devices (GPIO, SD-card, LCD, etc) that likely can
> be enabled and used. For this, the sysctl driver will need further
> improvements as each device uses a different clock. To share the fun,
> supporting these is left as an exercise for the hobbyist and hackers
> interested in this SoC/boards :)
>
> Christoph Hellwig (2):
>   riscv: Add Kendryte K210 SoC support
>   riscv: create a loader.bin for the kendryte kflash.py tool
>
> Damien Le Moal (8):
>   riscv: Fix gitignore
>   riscv: Force flat memory model with no-mmu
>   riscv: Unaligned load/store handling for M_MODE
>   riscv: Add BUILTIN_DTB support
>   riscv: Add SOC early init support
>   riscv: Select required drivers for Kendryte SOC
>   riscv: Add Kendryte K210 device tree
>   riscv: Kendryte K210 default config
>
>  arch/riscv/Kbuild                       |   1 +
>  arch/riscv/Kconfig                      |  19 ++
>  arch/riscv/Kconfig.socs                 |  10 +
>  arch/riscv/Makefile                     |   4 +-
>  arch/riscv/boot/.gitignore              |   2 +
>  arch/riscv/boot/Makefile                |   3 +
>  arch/riscv/boot/dts/Makefile            |   5 +
>  arch/riscv/boot/dts/kendryte/Makefile   |   2 +
>  arch/riscv/boot/dts/kendryte/k210.dts   |  23 ++
>  arch/riscv/boot/dts/kendryte/k210.dtsi  | 123 ++++++++
>  arch/riscv/configs/nommu_k210_defconfig |  68 +++++
>  arch/riscv/include/asm/soc.h            |  23 ++
>  arch/riscv/kernel/Makefile              |   3 +-
>  arch/riscv/kernel/head.S                |   1 +
>  arch/riscv/kernel/setup.c               |   6 +
>  arch/riscv/kernel/soc.c                 |  28 ++
>  arch/riscv/kernel/traps.c               |  27 +-
>  arch/riscv/kernel/traps_misaligned.c    | 371 ++++++++++++++++++++++++
>  arch/riscv/kernel/vmlinux.lds.S         |   6 +
>  arch/riscv/mm/init.c                    |   4 +
>  drivers/soc/Kconfig                     |   1 +
>  drivers/soc/Makefile                    |   1 +
>  drivers/soc/kendryte/Kconfig            |  14 +
>  drivers/soc/kendryte/Makefile           |   3 +
>  drivers/soc/kendryte/k210-sysctl.c      | 245 ++++++++++++++++
>  25 files changed, 987 insertions(+), 6 deletions(-)
>  create mode 100644 arch/riscv/boot/dts/kendryte/Makefile
>  create mode 100644 arch/riscv/boot/dts/kendryte/k210.dts
>  create mode 100644 arch/riscv/boot/dts/kendryte/k210.dtsi
>  create mode 100644 arch/riscv/configs/nommu_k210_defconfig
>  create mode 100644 arch/riscv/include/asm/soc.h
>  create mode 100644 arch/riscv/kernel/soc.c
>  create mode 100644 arch/riscv/kernel/traps_misaligned.c
>  create mode 100644 drivers/soc/kendryte/Kconfig
>  create mode 100644 drivers/soc/kendryte/Makefile
>  create mode 100644 drivers/soc/kendryte/k210-sysctl.c
>
> --
> 2.24.1
>
>

Hi Damien, I've tested the patches on v5.5.0 and it boots perfectly on
my MaixGo board. I used the provided initramfs.cpio as the payload and
got to the busybox prompt.

While trying to build my own busybox, I got a few problems. I've
checked-out your tree and copied the toolchain. Then when building
busybox with your minimal config, I got this error:

  LINK    busybox_unstripped
Your linker does not support --sort-section,alignment
Your linker does not support --sort-common
Your linker does not support -Wl,--gc-sections
Trying libraries: m rt
Failed: -Wl,--start-group  -lm -lrt  -Wl,--end-group
Output of:
/opt/riscv64-uclibc/bin/riscv64-linux-gcc -Wall -Wshadow
-Wwrite-strings -Wundef -Wstrict-prototypes -Wunused
-Wunused-parameter -Wunused-function -Wunused-value
-Wmissing-prototypes -Wmissing-declarations -Wno-format-security
-Wdeclaration-after-statement -Wold-style-definition -finline-limit=0
-fno-builtin-strlen -fomit-frame-pointer -ffunction-sections
-fdata-sections -fno-guess-branch-probability -funsigned-char
-static-libgcc -falign-functions=1 -falign-jumps=1 -falign-labels=1
-falign-loops=1 -fno-unwind-tables -fno-asynchronous-unwind-tables
-fno-builtin-printf -Wno-string-plus-int -Wno-constant-logical-operand
-Os -Os -fPIC --sysroot=/opt/riscv64-uclibc/riscv64-buildroot-linux-uclibc/sysroot/
-static -Os -static -Wl,-elf2flt=-r -o busybox_unstripped
-Wl,--start-group applets/built-in.o archival/lib.a
archival/libarchive/lib.a console-tools/lib.a coreutils/lib.a
coreutils/libcoreutils/lib.a debianutils/lib.a klibc-utils/lib.a
e2fsprogs/lib.a editors/lib.a findutils/lib.a init/lib.a libbb/lib.a
libpwdgrp/lib.a loginutils/lib.a mailutils/lib.a miscutils/lib.a
modutils/lib.a networking/lib.a networking/libiproute/lib.a
networking/udhcp/lib.a printutils/lib.a procps/lib.a runit/lib.a
selinux/lib.a shell/lib.a sysklogd/lib.a util-linux/lib.a
util-linux/volume_id/lib.a archival/built-in.o
archival/libarchive/built-in.o console-tools/built-in.o
coreutils/built-in.o coreutils/libcoreutils/built-in.o
debianutils/built-in.o klibc-utils/built-in.o e2fsprogs/built-in.o
editors/built-in.o findutils/built-in.o init/built-in.o
libbb/built-in.o libpwdgrp/built-in.o loginutils/built-in.o
mailutils/built-in.o miscutils/built-in.o modutils/built-in.o
networking/built-in.o networking/libiproute/built-in.o
networking/udhcp/built-in.o printutils/built-in.o procps/built-in.o
runit/built-in.o selinux/built-in.o shell/built-in.o
sysklogd/built-in.o util-linux/built-in.o
util-linux/volume_id/built-in.o -Wl,--end-group -Wl,--start-group -lm
-lrt -Wl,--end-group
==========
/opt/riscv64-uclibc/riscv64-buildroot-linux-uclibc/bin/ld.real: cannot
find crt1.o: No such file or directory
/opt/riscv64-uclibc/riscv64-buildroot-linux-uclibc/bin/ld.real: cannot
find crti.o: No such file or directory
/opt/riscv64-uclibc/riscv64-buildroot-linux-uclibc/bin/ld.real: cannot
find crtbeginT.o: No such file or directory
collect2: error: ld returned 1 exit status
Note: if build needs additional libraries, put them in CONFIG_EXTRA_LDLIBS.
Example: CONFIG_EXTRA_LDLIBS="pthread dl tirpc audit pam"
make: *** [Makefile:718: busybox_unstripped] Error 1

Then I've built the complete buildroot toolchain and replaced it on /opt.

Then I've proceeded to "make SKIP_STRIP=y" and "make install" and it
built fine. I got into _install dir and ran: "find . | sudo cpio -H
newc --create --verbose > ../k210.cpio" to generate the cpio. All this
with the

Rebuilt the kernel with it but when booting, I got this error:

[    0.259289] Run /sbin/init as init process
[    0.263480] Run /etc/init as init process
[    0.267453] Run /bin/init as init process
[    0.286973] Kernel panic - not syncing: Attempted to kill init!
exitcode=0x00000000
[    0.293869] CPU: 1 PID: 1 Comm: sh Not tainted 5.5.0-dirty #19
[    0.299673] Call Trace:
[    0.302109] [<00000000800401f6>] 0x00000000800401f6
[    0.306969] [<000000008004033a>] 0x000000008004033a
[    0.311831] [<0000000080111abe>] 0x0000000080111abe
[    0.316693] [<000000008004340e>] 0x000000008004340e
[    0.321556] [<0000000080045402>] 0x0000000080045402
[    0.326417] [<0000000080045898>] 0x0000000080045898
[    0.331279] [<000000008003f1d2>] 0x000000008003f1d2
[    0.336142] SMP: stopping secondary CPUs
[    0.340065] ---[ end Kernel panic - not syncing: Attempted to kill
init! exitcode=0x00000000 ]---

I also tried with a gzipped cpio but got the "Kernel panic - not
syncing: no cpio magic" error.

What's the right way to create the cpio initramfs? I want to customize
it a little.

Thanks.

Damien Le Moal Feb. 15, 2020, 2:02 a.m. UTC | #2

On 2020/02/15 0:06, Carlos Eduardo de Paula wrote:
> Hi Damien, I've tested the patches on v5.5.0 and it boots perfectly on
> my MaixGo board. I used the provided initramfs.cpio as the payload and
> got to the busybox prompt.

Great !

> 
> While trying to build my own busybox, I got a few problems. I've
> checked-out your tree and copied the toolchain. Then when building
> busybox with your minimal config, I got this error:
> 
>   LINK    busybox_unstripped
> Your linker does not support --sort-section,alignment
> Your linker does not support --sort-common
> Your linker does not support -Wl,--gc-sections
> Trying libraries: m rt
> Failed: -Wl,--start-group  -lm -lrt  -Wl,--end-group
> Output of:
> /opt/riscv64-uclibc/bin/riscv64-linux-gcc -Wall -Wshadow
> -Wwrite-strings -Wundef -Wstrict-prototypes -Wunused
> -Wunused-parameter -Wunused-function -Wunused-value
> -Wmissing-prototypes -Wmissing-declarations -Wno-format-security
> -Wdeclaration-after-statement -Wold-style-definition -finline-limit=0
> -fno-builtin-strlen -fomit-frame-pointer -ffunction-sections
> -fdata-sections -fno-guess-branch-probability -funsigned-char
> -static-libgcc -falign-functions=1 -falign-jumps=1 -falign-labels=1
> -falign-loops=1 -fno-unwind-tables -fno-asynchronous-unwind-tables
> -fno-builtin-printf -Wno-string-plus-int -Wno-constant-logical-operand
> -Os -Os -fPIC --sysroot=/opt/riscv64-uclibc/riscv64-buildroot-linux-uclibc/sysroot/
> -static -Os -static -Wl,-elf2flt=-r -o busybox_unstripped
> -Wl,--start-group applets/built-in.o archival/lib.a
> archival/libarchive/lib.a console-tools/lib.a coreutils/lib.a
> coreutils/libcoreutils/lib.a debianutils/lib.a klibc-utils/lib.a
> e2fsprogs/lib.a editors/lib.a findutils/lib.a init/lib.a libbb/lib.a
> libpwdgrp/lib.a loginutils/lib.a mailutils/lib.a miscutils/lib.a
> modutils/lib.a networking/lib.a networking/libiproute/lib.a
> networking/udhcp/lib.a printutils/lib.a procps/lib.a runit/lib.a
> selinux/lib.a shell/lib.a sysklogd/lib.a util-linux/lib.a
> util-linux/volume_id/lib.a archival/built-in.o
> archival/libarchive/built-in.o console-tools/built-in.o
> coreutils/built-in.o coreutils/libcoreutils/built-in.o
> debianutils/built-in.o klibc-utils/built-in.o e2fsprogs/built-in.o
> editors/built-in.o findutils/built-in.o init/built-in.o
> libbb/built-in.o libpwdgrp/built-in.o loginutils/built-in.o
> mailutils/built-in.o miscutils/built-in.o modutils/built-in.o
> networking/built-in.o networking/libiproute/built-in.o
> networking/udhcp/built-in.o printutils/built-in.o procps/built-in.o
> runit/built-in.o selinux/built-in.o shell/built-in.o
> sysklogd/built-in.o util-linux/built-in.o
> util-linux/volume_id/built-in.o -Wl,--end-group -Wl,--start-group -lm
> -lrt -Wl,--end-group
> ==========
> /opt/riscv64-uclibc/riscv64-buildroot-linux-uclibc/bin/ld.real: cannot
> find crt1.o: No such file or directory
> /opt/riscv64-uclibc/riscv64-buildroot-linux-uclibc/bin/ld.real: cannot
> find crti.o: No such file or directory
> /opt/riscv64-uclibc/riscv64-buildroot-linux-uclibc/bin/ld.real: cannot
> find crtbeginT.o: No such file or directory
> collect2: error: ld returned 1 exit status
> Note: if build needs additional libraries, put them in CONFIG_EXTRA_LDLIBS.
> Example: CONFIG_EXTRA_LDLIBS="pthread dl tirpc audit pam"
> make: *** [Makefile:718: busybox_unstripped] Error 1

Weird... librt is not needed normally and is actually not generated by the
toolchain due to the absence of native thread support I think. Maybe it was
needed due to some option selection in busybox you did ?

> Then I've built the complete buildroot toolchain and replaced it on /opt.
> 
> Then I've proceeded to "make SKIP_STRIP=y" and "make install" and it
> built fine. I got into _install dir and ran: "find . | sudo cpio -H
> newc --create --verbose > ../k210.cpio" to generate the cpio. All this
> with the
> 
> Rebuilt the kernel with it but when booting, I got this error:
> 
> [    0.259289] Run /sbin/init as init process
> [    0.263480] Run /etc/init as init process
> [    0.267453] Run /bin/init as init process
> [    0.286973] Kernel panic - not syncing: Attempted to kill init!
> exitcode=0x00000000
> [    0.293869] CPU: 1 PID: 1 Comm: sh Not tainted 5.5.0-dirty #19
> [    0.299673] Call Trace:
> [    0.302109] [<00000000800401f6>] 0x00000000800401f6
> [    0.306969] [<000000008004033a>] 0x000000008004033a
> [    0.311831] [<0000000080111abe>] 0x0000000080111abe
> [    0.316693] [<000000008004340e>] 0x000000008004340e
> [    0.321556] [<0000000080045402>] 0x0000000080045402
> [    0.326417] [<0000000080045898>] 0x0000000080045898
> [    0.331279] [<000000008003f1d2>] 0x000000008003f1d2
> [    0.336142] SMP: stopping secondary CPUs
> [    0.340065] ---[ end Kernel panic - not syncing: Attempted to kill
> init! exitcode=0x00000000 ]---

The busybox _install directory is not enough on its own to become an
initramfs tree. You need to add some stuff to it. I use the following
script to build a simple one with _install as a base:

#!/bin/bash

if [ $# != 2 ]; then
	echo "Usage: $0 <busybox install dir> <cpio img path>"
	exit 1
fi

# Prepare
cd $1
mkdir dev sys proc tmp root etc
mkdir dev/pts dev/shm

cd dev
sudo mknod -m 622 console c 5 1
sudo mknod -m 666 null c 1 3
sudo mknod -m 666 zero c 1 5
sudo mknod -m 666 ptmx c 5 2
sudo mknod -m 666 tty c 5 0
sudo mknod -m 444 random c 1 8
sudo mknod -m 444 urandom c 1 9
sudo mknod -m 666 ttySIF0 c 4 64
sudo mknod -m 666 tty0 c 4 0
sudo chown root:tty {console,ptmx,tty}
cd ..

# Create image file
echo "Creating cpio image"

find . | cpio -H newc -o > $2

> 
> I also tried with a gzipped cpio but got the "Kernel panic - not
> syncing: no cpio magic" error.
> 
> What's the right way to create the cpio initramfs? I want to customize
> it a little.

You also need to copy an init script under /bin or /sbin in the _install
directory before running the above script. The one with the precompiled
image I wrote is simply:

#!/bin/sh

echo ""
echo "-----------------------------"
echo "| Kendryte K210 NOMMU Linux |"
echo "-----------------------------"

echo "Mounting /proc"
mount -t proc proc /proc

echo "Starting shell"
exec /bin/sh


At least for the config I use with busybox, I do not get one automatically.
I tend to not use the default busybox init stuff to keep control over what
is done and keep things small overall. But since things are now starting to
work well, we can start experimenting other configs for busybox (e.g. a
more complete one).

Cheers.

> 
> Thanks.
>

Carlos de Paula Feb. 17, 2020, 1:28 p.m. UTC | #3

Hi Damien, thanks for pointing this out. I've created the directory
structure and their permissions and was able to build a custom
busybox. I'll now check on how to use the buildroot generated cpio on
it.

On Fri, Feb 14, 2020 at 11:02 PM Damien Le Moal <Damien.LeMoal@wdc.com> wrote:
>
> On 2020/02/15 0:06, Carlos Eduardo de Paula wrote:
> > Hi Damien, I've tested the patches on v5.5.0 and it boots perfectly on
> > my MaixGo board. I used the provided initramfs.cpio as the payload and
> > got to the busybox prompt.
>
> Great !
>
> >
> > While trying to build my own busybox, I got a few problems. I've
> > checked-out your tree and copied the toolchain. Then when building
> > busybox with your minimal config, I got this error:
> >
> >   LINK    busybox_unstripped
> > Your linker does not support --sort-section,alignment
> > Your linker does not support --sort-common
> > Your linker does not support -Wl,--gc-sections
> > Trying libraries: m rt
> > Failed: -Wl,--start-group  -lm -lrt  -Wl,--end-group
> > Output of:
> > /opt/riscv64-uclibc/bin/riscv64-linux-gcc -Wall -Wshadow
> > -Wwrite-strings -Wundef -Wstrict-prototypes -Wunused
> > -Wunused-parameter -Wunused-function -Wunused-value
> > -Wmissing-prototypes -Wmissing-declarations -Wno-format-security
> > -Wdeclaration-after-statement -Wold-style-definition -finline-limit=0
> > -fno-builtin-strlen -fomit-frame-pointer -ffunction-sections
> > -fdata-sections -fno-guess-branch-probability -funsigned-char
> > -static-libgcc -falign-functions=1 -falign-jumps=1 -falign-labels=1
> > -falign-loops=1 -fno-unwind-tables -fno-asynchronous-unwind-tables
> > -fno-builtin-printf -Wno-string-plus-int -Wno-constant-logical-operand
> > -Os -Os -fPIC --sysroot=/opt/riscv64-uclibc/riscv64-buildroot-linux-uclibc/sysroot/
> > -static -Os -static -Wl,-elf2flt=-r -o busybox_unstripped
> > -Wl,--start-group applets/built-in.o archival/lib.a
> > archival/libarchive/lib.a console-tools/lib.a coreutils/lib.a
> > coreutils/libcoreutils/lib.a debianutils/lib.a klibc-utils/lib.a
> > e2fsprogs/lib.a editors/lib.a findutils/lib.a init/lib.a libbb/lib.a
> > libpwdgrp/lib.a loginutils/lib.a mailutils/lib.a miscutils/lib.a
> > modutils/lib.a networking/lib.a networking/libiproute/lib.a
> > networking/udhcp/lib.a printutils/lib.a procps/lib.a runit/lib.a
> > selinux/lib.a shell/lib.a sysklogd/lib.a util-linux/lib.a
> > util-linux/volume_id/lib.a archival/built-in.o
> > archival/libarchive/built-in.o console-tools/built-in.o
> > coreutils/built-in.o coreutils/libcoreutils/built-in.o
> > debianutils/built-in.o klibc-utils/built-in.o e2fsprogs/built-in.o
> > editors/built-in.o findutils/built-in.o init/built-in.o
> > libbb/built-in.o libpwdgrp/built-in.o loginutils/built-in.o
> > mailutils/built-in.o miscutils/built-in.o modutils/built-in.o
> > networking/built-in.o networking/libiproute/built-in.o
> > networking/udhcp/built-in.o printutils/built-in.o procps/built-in.o
> > runit/built-in.o selinux/built-in.o shell/built-in.o
> > sysklogd/built-in.o util-linux/built-in.o
> > util-linux/volume_id/built-in.o -Wl,--end-group -Wl,--start-group -lm
> > -lrt -Wl,--end-group
> > ==========
> > /opt/riscv64-uclibc/riscv64-buildroot-linux-uclibc/bin/ld.real: cannot
> > find crt1.o: No such file or directory
> > /opt/riscv64-uclibc/riscv64-buildroot-linux-uclibc/bin/ld.real: cannot
> > find crti.o: No such file or directory
> > /opt/riscv64-uclibc/riscv64-buildroot-linux-uclibc/bin/ld.real: cannot
> > find crtbeginT.o: No such file or directory
> > collect2: error: ld returned 1 exit status
> > Note: if build needs additional libraries, put them in CONFIG_EXTRA_LDLIBS.
> > Example: CONFIG_EXTRA_LDLIBS="pthread dl tirpc audit pam"
> > make: *** [Makefile:718: busybox_unstripped] Error 1
>
> Weird... librt is not needed normally and is actually not generated by the
> toolchain due to the absence of native thread support I think. Maybe it was
> needed due to some option selection in busybox you did ?
>
> > Then I've built the complete buildroot toolchain and replaced it on /opt.
> >
> > Then I've proceeded to "make SKIP_STRIP=y" and "make install" and it
> > built fine. I got into _install dir and ran: "find . | sudo cpio -H
> > newc --create --verbose > ../k210.cpio" to generate the cpio. All this
> > with the
> >
> > Rebuilt the kernel with it but when booting, I got this error:
> >
> > [    0.259289] Run /sbin/init as init process
> > [    0.263480] Run /etc/init as init process
> > [    0.267453] Run /bin/init as init process
> > [    0.286973] Kernel panic - not syncing: Attempted to kill init!
> > exitcode=0x00000000
> > [    0.293869] CPU: 1 PID: 1 Comm: sh Not tainted 5.5.0-dirty #19
> > [    0.299673] Call Trace:
> > [    0.302109] [<00000000800401f6>] 0x00000000800401f6
> > [    0.306969] [<000000008004033a>] 0x000000008004033a
> > [    0.311831] [<0000000080111abe>] 0x0000000080111abe
> > [    0.316693] [<000000008004340e>] 0x000000008004340e
> > [    0.321556] [<0000000080045402>] 0x0000000080045402
> > [    0.326417] [<0000000080045898>] 0x0000000080045898
> > [    0.331279] [<000000008003f1d2>] 0x000000008003f1d2
> > [    0.336142] SMP: stopping secondary CPUs
> > [    0.340065] ---[ end Kernel panic - not syncing: Attempted to kill
> > init! exitcode=0x00000000 ]---
>
> The busybox _install directory is not enough on its own to become an
> initramfs tree. You need to add some stuff to it. I use the following
> script to build a simple one with _install as a base:
>
> #!/bin/bash
>
> if [ $# != 2 ]; then
>         echo "Usage: $0 <busybox install dir> <cpio img path>"
>         exit 1
> fi
>
> # Prepare
> cd $1
> mkdir dev sys proc tmp root etc
> mkdir dev/pts dev/shm
>
> cd dev
> sudo mknod -m 622 console c 5 1
> sudo mknod -m 666 null c 1 3
> sudo mknod -m 666 zero c 1 5
> sudo mknod -m 666 ptmx c 5 2
> sudo mknod -m 666 tty c 5 0
> sudo mknod -m 444 random c 1 8
> sudo mknod -m 444 urandom c 1 9
> sudo mknod -m 666 ttySIF0 c 4 64
> sudo mknod -m 666 tty0 c 4 0
> sudo chown root:tty {console,ptmx,tty}
> cd ..
>
> # Create image file
> echo "Creating cpio image"
>
> find . | cpio -H newc -o > $2
>
> >
> > I also tried with a gzipped cpio but got the "Kernel panic - not
> > syncing: no cpio magic" error.
> >
> > What's the right way to create the cpio initramfs? I want to customize
> > it a little.
>
> You also need to copy an init script under /bin or /sbin in the _install
> directory before running the above script. The one with the precompiled
> image I wrote is simply:
>
> #!/bin/sh
>
> echo ""
> echo "-----------------------------"
> echo "| Kendryte K210 NOMMU Linux |"
> echo "-----------------------------"
>
> echo "Mounting /proc"
> mount -t proc proc /proc
>
> echo "Starting shell"
> exec /bin/sh
>
>
> At least for the config I use with busybox, I do not get one automatically.
> I tend to not use the default busybox init stuff to keep control over what
> is done and keep things small overall. But since things are now starting to
> work well, we can start experimenting other configs for busybox (e.g. a
> more complete one).
>
> Cheers.
>
> >
> > Thanks.
> >
>
>
> --
> Damien Le Moal
> Western Digital Research

Carlos de Paula Feb. 26, 2020, 9:31 p.m. UTC | #4

> >
> > >
> > > Thanks.
> > >
> >
> >
> > --
> > Damien Le Moal
> > Western Digital Research
>

I've integrated all required changes into a buildroot fork from your
repo where it's possible to build the complete image (Kernel + Patches
+ Busybox iniramfs) in a single place and there is no need to place
the toolchain outside, build the kernel and etc.

The changes are in https://github.com/carlosedp/riscv64-nommu-buildroot.

@Damien Le Moal I can send you a PR if you want.

Carlos

--

Damien Le Moal Feb. 27, 2020, 2:18 a.m. UTC | #5

On 2020/02/27 6:32, Carlos Eduardo de Paula wrote:
>>>
>>>>
>>>> Thanks.
>>>>
>>>
>>>
>>> --
>>> Damien Le Moal
>>> Western Digital Research
>>
> 
> I've integrated all required changes into a buildroot fork from your
> repo where it's possible to build the complete image (Kernel + Patches
> + Busybox iniramfs) in a single place and there is no need to place
> the toolchain outside, build the kernel and etc.
> 
> The changes are in https://github.com/carlosedp/riscv64-nommu-buildroot.
> 
> @Damien Le Moal I can send you a PR if you want.

Yes, please do ! I need to prepare a v2 of the patchset. I can integrate the v2
patches with your fixes for everybody to test.

Thanks !

> 
> Carlos
> 
> --
> ________________________________________
> Carlos Eduardo de Paula
> me@carlosedp.com
> http://carlosedp.com
> http://twitter.com/carlosedp
> Linkedin
> ________________________________________
> 
>

Sean Anderson Feb. 28, 2020, 8:32 p.m. UTC | #6

Hi,

When booting from U-Boot I get an OOM. Perhaps this is related to the
second cpu not coming up?

=> go 80000000
## Starting application at 0x80000000 ...
[    0.000000] Linux version 5.6.0-rc1-00054-gd32122774dab (sean@godwin) (gcc version 9.2.0 (GCC)) #12 SMP Fri Feb 28 14:34:45 EST 2020
[    0.000000] earlycon: sifive0 at MMIO 0x0000000038000000 (options '')
[    0.000000] printk: bootconsole [sifive0] enabled
[    0.000000] initrd not found or empty - disabling initrd
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000080000000-0x00000000807fffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080000000-0x00000000807fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x00000000807fffff]
[    0.000000] elf_hwcap is 0x112d
[    0.000000] percpu: max_distance=0x18000 too large for vmalloc space 0x0
[    0.000000] percpu: Embedded 12 pages/cpu s18272 r0 d30880 u49152
[    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2020
[    0.000000] Kernel command line: earlycon console=ttySIF0
[    0.000000] Dentry cache hash table entries: 1024 (order: 1, 8192 bytes, linear)
[    0.000000] Inode-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
[    0.000000] Sorting __ex_table...
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 5528K/8192K available (918K kernel code, 106K rwdata, 166K rodata, 1129K init, 91K bss, 2664K reserved, 0K cma-reserved)
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] NR_IRQS: 0, nr_irqs: 0, preallocated irqs: 0
[    0.000000] plic: mapped 65 interrupts with 2 handlers for 4 contexts.
[    0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x3990be68b, max_idle_ns: 881590404272 ns
[    0.000015] sched_clock: 64 bits at 7MHz, resolution 128ns, wraps every 4398046511054ns
[    0.008238] Console: colour dummy device 80x25
[    0.012477] Calibrating delay loop (skipped), value calculated using timer frequency.. 15.60 BogoMIPS (lpj=31200)
[    0.022684] pid_max: default: 4096 minimum: 301
[    0.027302] Mount-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
[    0.034423] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
[    0.044826] rcu: Hierarchical SRCU implementation.
[    0.049623] smp: Bringing up secondary CPUs ...
[    1.083970] CPU1: failed to come online
[    1.087079] smp: Brought up 1 node, 1 CPU
[    1.092006] devtmpfs: initialized
[    1.098586] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    1.107609] futex hash table entries: 16 (order: -2, 1024 bytes, linear)
[    1.115619] Kendryte K210 SoC sysctl
[    1.129362] clocksource: Switched to clocksource riscv_clocksource
[    1.385690] swapper/0: page allocation failure: order:9, mode:0x100cc2(GFP_HIGHUSER), nodemask=(null)
[    1.394224] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.6.0-rc1-00054-gd32122774dab #12
[    1.402129] Call Trace:
[    1.404565] [<000000008011c2f2>] 0x000000008011c2f2
[    1.409426] [<000000008011c436>] 0x000000008011c436
[    1.414287] [<00000000801ed86e>] 0x00000000801ed86e
[    1.419150] [<00000000801740c0>] 0x00000000801740c0
[    1.424012] [<00000000801743d4>] 0x00000000801743d4
[    1.428874] [<00000000801746da>] 0x00000000801746da
[    1.433736] [<00000000801ababe>] 0x00000000801ababe
[    1.438598] [<00000000801abbf2>] 0x00000000801abbf2
[    1.443460] [<000000008018b06a>] 0x000000008018b06a
[    1.448322] [<0000000080176de0>] 0x0000000080176de0
[    1.453184] [<0000000080176fd2>] 0x0000000080176fd2
[    1.458047] [<0000000080001b8a>] 0x0000000080001b8a
[    1.462909] [<000000008000145a>] 0x000000008000145a
[    1.467771] [<00000000800014b0>] 0x00000000800014b0
[    1.472633] [<000000008000e7cc>] 0x000000008000e7cc
[    1.477495] [<000000008000e80c>] 0x000000008000e80c
[    1.482357] [<0000000080001e44>] 0x0000000080001e44
[    1.487219] [<0000000080000a80>] 0x0000000080000a80
[    1.492081] [<0000000080000ce4>] 0x0000000080000ce4
[    1.496943] [<00000000801fd934>] 0x00000000801fd934
[    1.501805] [<000000008011b304>] 0x000000008011b304
[    1.506918] Mem-Info:
[    1.508957] active_anon:0 inactive_anon:0 isolated_anon:0
[    1.508957]  active_file:0 inactive_file:0 isolated_file:0
[    1.508957]  unevictable:315 dirty:0 writeback:0 unstable:0
[    1.508957]  slab_reclaimable:0 slab_unreclaimable:215
[    1.508957]  mapped:0 shmem:0 pagetables:0 bounce:0
[    1.508957]  free:795 free_pcp:0 free_cma:0
[    1.539593] Node 0 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:1260kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[    1.560939] DMA32 free:3180kB min:296kB low:368kB high:440kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:1260kB writepending:0kB present:8192kB managed:5528kB mlocked:0kB kernel_stack:168kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[    1.588696] lowmem_reserve[]: 0 0 0
[    1.592118] DMA32: 1*4kB (U) 1*8kB (M) 0*16kB 1*32kB (U) 1*64kB (U) 2*128kB (UM) 1*256kB (U) 1*512kB (M) 2*1024kB (UM) 0*2048kB 0*4096kB = 3180kB
[    1.605162] 328 total pagecache pages
[    1.608788] 2048 pages RAM
[    1.611479] 0 pages HighMem/MovableOnly
[    1.615299] 666 pages reserved
[    1.743951] swapper/0 invoked oom-killer: gfp_mask=0x100cc2(GFP_HIGHUSER), order=0, oom_score_adj=0
[    1.752280] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.6.0-rc1-00054-gd32122774dab #12
[    1.760209] Call Trace:
[    1.762645] [<000000008011c2f2>] 0x000000008011c2f2
[    1.767506] [<000000008011c436>] 0x000000008011c436
[    1.772368] [<00000000801ed86e>] 0x00000000801ed86e
[    1.777230] [<00000000801634e2>] 0x00000000801634e2
[    1.782092] [<00000000801633c6>] 0x00000000801633c6
[    1.786954] [<000000008017451e>] 0x000000008017451e
[    1.791816] [<00000000801746da>] 0x00000000801746da
[    1.796678] [<0000000080161648>] 0x0000000080161648
[    1.801540] [<000000008016241e>] 0x000000008016241e
[    1.806402] [<0000000080192fc6>] 0x0000000080192fc6
[    1.811264] [<00000000801624a6>] 0x00000000801624a6
[    1.816127] [<000000008016262c>] 0x000000008016262c
[    1.820989] [<000000008016267e>] 0x000000008016267e
[    1.825851] [<0000000080178178>] 0x0000000080178178
[    1.830713] [<00000000801781c0>] 0x00000000801781c0
[    1.835575] [<0000000080178b5c>] 0x0000000080178b5c
[    1.840437] [<0000000080178c82>] 0x0000000080178c82
[    1.845299] [<0000000080001678>] 0x0000000080001678
[    1.850161] [<0000000080001724>] 0x0000000080001724
[    1.855023] [<000000008000145a>] 0x000000008000145a
[    1.859885] [<00000000800014b0>] 0x00000000800014b0
[    1.864748] [<000000008000e7cc>] 0x000000008000e7cc
[    1.869609] [<000000008000e80c>] 0x000000008000e80c
[    1.874472] [<0000000080001e44>] 0x0000000080001e44
[    1.879334] [<0000000080000a80>] 0x0000000080000a80
[    1.884196] [<0000000080000ce4>] 0x0000000080000ce4
[    1.889058] [<00000000801fd934>] 0x00000000801fd934
[    1.893920] [<000000008011b304>] 0x000000008011b304
[    1.899086] Mem-Info:
[    1.901072] active_anon:0 inactive_anon:0 isolated_anon:0
[    1.901072]  active_file:0 inactive_file:0 isolated_file:0
[    1.901072]  unevictable:705 dirty:0 writeback:0 unstable:0
[    1.901072]  slab_reclaimable:0 slab_unreclaimable:215
[    1.901072]  mapped:0 shmem:0 pagetables:0 bounce:0
[    1.901072]  free:417 free_pcp:0 free_cma:0
[    1.931708] Node 0 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:2820kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[    1.953051] DMA32 free:1668kB min:4392kB low:4464kB high:4536kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:2820kB writepending:0kB present:8192kB managed:5528kB mlocked:0kB kernel_stack:168kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[    1.981067] lowmem_reserve[]: 0 0 0
[    1.984492] DMA32: 1*4kB (U) 0*8kB 0*16kB 0*32kB 0*64kB 1*128kB (U) 0*256kB 1*512kB (U) 1*1024kB (U) 0*2048kB 0*4096kB = 1668kB
[    1.995970] 704 total pagecache pages
[    1.999599] 2048 pages RAM
[    2.002291] 0 pages HighMem/MovableOnly
[    2.006110] 666 pages reserved
[    2.009131] Tasks state (memory values in pages):
[    2.013848] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
[    2.022450] Out of memory and no killable processes...
[    2.027562] Kernel panic - not syncing: System is deadlocked on memory
[    2.034062] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.6.0-rc1-00054-gd32122774dab #12
[    2.042036] Call Trace:
[    2.044472] [<000000008011c2f2>] 0x000000008011c2f2
[    2.049333] [<000000008011c436>] 0x000000008011c436
[    2.054195] [<00000000801ed86e>] 0x00000000801ed86e
[    2.059057] [<000000008011f4d8>] 0x000000008011f4d8
[    2.063919] [<00000000801633ea>] 0x00000000801633ea
[    2.068782] [<000000008017451e>] 0x000000008017451e
[    2.073644] [<00000000801746da>] 0x00000000801746da
[    2.078506] [<0000000080161648>] 0x0000000080161648
[    2.083368] [<000000008016241e>] 0x000000008016241e
[    2.088230] [<0000000080192fc6>] 0x0000000080192fc6
[    2.093092] [<00000000801624a6>] 0x00000000801624a6
[    2.097954] [<000000008016262c>] 0x000000008016262c
[    2.102816] [<000000008016267e>] 0x000000008016267e
[    2.107678] [<0000000080178178>] 0x0000000080178178
[    2.112541] [<00000000801781c0>] 0x00000000801781c0
[    2.117403] [<0000000080178b5c>] 0x0000000080178b5c
[    2.122265] [<0000000080178c82>] 0x0000000080178c82
[    2.127127] [<0000000080001678>] 0x0000000080001678
[    2.131989] [<0000000080001724>] 0x0000000080001724
[    2.136851] [<000000008000145a>] 0x000000008000145a
[    2.141713] [<00000000800014b0>] 0x00000000800014b0
[    2.146575] [<000000008000e7cc>] 0x000000008000e7cc
[    2.151437] [<000000008000e80c>] 0x000000008000e80c
[    2.156299] [<0000000080001e44>] 0x0000000080001e44
[    2.161162] [<0000000080000a80>] 0x0000000080000a80
[    2.166024] [<0000000080000ce4>] 0x0000000080000ce4
[    2.170886] [<00000000801fd934>] 0x00000000801fd934
[    2.175748] [<000000008011b304>] 0x000000008011b304
[    2.180624] ---[ end Kernel panic - not syncing: System is deadlocked on memory ]---

Damien Le Moal March 2, 2020, 3:01 a.m. UTC | #7

On 2020/02/29 5:32, Sean Anderson wrote:
> Hi,
> 
> When booting from U-Boot I get an OOM. Perhaps this is related to the
> second cpu not coming up?

Unlikely. It looks like your user space needs 2MB per shell (order 9
allocation). Since you have only 5.5MB free, that may explain the allocation
failure (if init is forking another shell especially).

For the second core not coming up, an IPI needs to be sent to the non-boot core
to wake it up. A Kendryte thing. U-boot should probably do it before jumping to
the kernel. I thought I had that in the kernel though. Will check again.

> 
> => go 80000000
> ## Starting application at 0x80000000 ...
> [    0.000000] Linux version 5.6.0-rc1-00054-gd32122774dab (sean@godwin) (gcc version 9.2.0 (GCC)) #12 SMP Fri Feb 28 14:34:45 EST 2020
> [    0.000000] earlycon: sifive0 at MMIO 0x0000000038000000 (options '')
> [    0.000000] printk: bootconsole [sifive0] enabled
> [    0.000000] initrd not found or empty - disabling initrd
> [    0.000000] Zone ranges:
> [    0.000000]   DMA32    [mem 0x0000000080000000-0x00000000807fffff]
> [    0.000000]   Normal   empty
> [    0.000000] Movable zone start for each node
> [    0.000000] Early memory node ranges
> [    0.000000]   node   0: [mem 0x0000000080000000-0x00000000807fffff]
> [    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x00000000807fffff]
> [    0.000000] elf_hwcap is 0x112d
> [    0.000000] percpu: max_distance=0x18000 too large for vmalloc space 0x0
> [    0.000000] percpu: Embedded 12 pages/cpu s18272 r0 d30880 u49152
> [    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2020
> [    0.000000] Kernel command line: earlycon console=ttySIF0
> [    0.000000] Dentry cache hash table entries: 1024 (order: 1, 8192 bytes, linear)
> [    0.000000] Inode-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
> [    0.000000] Sorting __ex_table...
> [    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
> [    0.000000] Memory: 5528K/8192K available (918K kernel code, 106K rwdata, 166K rodata, 1129K init, 91K bss, 2664K reserved, 0K cma-reserved)
> [    0.000000] rcu: Hierarchical RCU implementation.
> [    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
> [    0.000000] NR_IRQS: 0, nr_irqs: 0, preallocated irqs: 0
> [    0.000000] plic: mapped 65 interrupts with 2 handlers for 4 contexts.
> [    0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
> [    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x3990be68b, max_idle_ns: 881590404272 ns
> [    0.000015] sched_clock: 64 bits at 7MHz, resolution 128ns, wraps every 4398046511054ns
> [    0.008238] Console: colour dummy device 80x25
> [    0.012477] Calibrating delay loop (skipped), value calculated using timer frequency.. 15.60 BogoMIPS (lpj=31200)
> [    0.022684] pid_max: default: 4096 minimum: 301
> [    0.027302] Mount-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
> [    0.034423] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
> [    0.044826] rcu: Hierarchical SRCU implementation.
> [    0.049623] smp: Bringing up secondary CPUs ...
> [    1.083970] CPU1: failed to come online
> [    1.087079] smp: Brought up 1 node, 1 CPU
> [    1.092006] devtmpfs: initialized
> [    1.098586] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
> [    1.107609] futex hash table entries: 16 (order: -2, 1024 bytes, linear)
> [    1.115619] Kendryte K210 SoC sysctl
> [    1.129362] clocksource: Switched to clocksource riscv_clocksource
> [    1.385690] swapper/0: page allocation failure: order:9, mode:0x100cc2(GFP_HIGHUSER), nodemask=(null)
> [    1.394224] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.6.0-rc1-00054-gd32122774dab #12
> [    1.402129] Call Trace:
> [    1.404565] [<000000008011c2f2>] 0x000000008011c2f2
> [    1.409426] [<000000008011c436>] 0x000000008011c436
> [    1.414287] [<00000000801ed86e>] 0x00000000801ed86e
> [    1.419150] [<00000000801740c0>] 0x00000000801740c0
> [    1.424012] [<00000000801743d4>] 0x00000000801743d4
> [    1.428874] [<00000000801746da>] 0x00000000801746da
> [    1.433736] [<00000000801ababe>] 0x00000000801ababe
> [    1.438598] [<00000000801abbf2>] 0x00000000801abbf2
> [    1.443460] [<000000008018b06a>] 0x000000008018b06a
> [    1.448322] [<0000000080176de0>] 0x0000000080176de0
> [    1.453184] [<0000000080176fd2>] 0x0000000080176fd2
> [    1.458047] [<0000000080001b8a>] 0x0000000080001b8a
> [    1.462909] [<000000008000145a>] 0x000000008000145a
> [    1.467771] [<00000000800014b0>] 0x00000000800014b0
> [    1.472633] [<000000008000e7cc>] 0x000000008000e7cc
> [    1.477495] [<000000008000e80c>] 0x000000008000e80c
> [    1.482357] [<0000000080001e44>] 0x0000000080001e44
> [    1.487219] [<0000000080000a80>] 0x0000000080000a80
> [    1.492081] [<0000000080000ce4>] 0x0000000080000ce4
> [    1.496943] [<00000000801fd934>] 0x00000000801fd934
> [    1.501805] [<000000008011b304>] 0x000000008011b304
> [    1.506918] Mem-Info:
> [    1.508957] active_anon:0 inactive_anon:0 isolated_anon:0
> [    1.508957]  active_file:0 inactive_file:0 isolated_file:0
> [    1.508957]  unevictable:315 dirty:0 writeback:0 unstable:0
> [    1.508957]  slab_reclaimable:0 slab_unreclaimable:215
> [    1.508957]  mapped:0 shmem:0 pagetables:0 bounce:0
> [    1.508957]  free:795 free_pcp:0 free_cma:0
> [    1.539593] Node 0 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:1260kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
> [    1.560939] DMA32 free:3180kB min:296kB low:368kB high:440kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:1260kB writepending:0kB present:8192kB managed:5528kB mlocked:0kB kernel_stack:168kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
> [    1.588696] lowmem_reserve[]: 0 0 0
> [    1.592118] DMA32: 1*4kB (U) 1*8kB (M) 0*16kB 1*32kB (U) 1*64kB (U) 2*128kB (UM) 1*256kB (U) 1*512kB (M) 2*1024kB (UM) 0*2048kB 0*4096kB = 3180kB
> [    1.605162] 328 total pagecache pages
> [    1.608788] 2048 pages RAM
> [    1.611479] 0 pages HighMem/MovableOnly
> [    1.615299] 666 pages reserved
> [    1.743951] swapper/0 invoked oom-killer: gfp_mask=0x100cc2(GFP_HIGHUSER), order=0, oom_score_adj=0
> [    1.752280] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.6.0-rc1-00054-gd32122774dab #12
> [    1.760209] Call Trace:
> [    1.762645] [<000000008011c2f2>] 0x000000008011c2f2
> [    1.767506] [<000000008011c436>] 0x000000008011c436
> [    1.772368] [<00000000801ed86e>] 0x00000000801ed86e
> [    1.777230] [<00000000801634e2>] 0x00000000801634e2
> [    1.782092] [<00000000801633c6>] 0x00000000801633c6
> [    1.786954] [<000000008017451e>] 0x000000008017451e
> [    1.791816] [<00000000801746da>] 0x00000000801746da
> [    1.796678] [<0000000080161648>] 0x0000000080161648
> [    1.801540] [<000000008016241e>] 0x000000008016241e
> [    1.806402] [<0000000080192fc6>] 0x0000000080192fc6
> [    1.811264] [<00000000801624a6>] 0x00000000801624a6
> [    1.816127] [<000000008016262c>] 0x000000008016262c
> [    1.820989] [<000000008016267e>] 0x000000008016267e
> [    1.825851] [<0000000080178178>] 0x0000000080178178
> [    1.830713] [<00000000801781c0>] 0x00000000801781c0
> [    1.835575] [<0000000080178b5c>] 0x0000000080178b5c
> [    1.840437] [<0000000080178c82>] 0x0000000080178c82
> [    1.845299] [<0000000080001678>] 0x0000000080001678
> [    1.850161] [<0000000080001724>] 0x0000000080001724
> [    1.855023] [<000000008000145a>] 0x000000008000145a
> [    1.859885] [<00000000800014b0>] 0x00000000800014b0
> [    1.864748] [<000000008000e7cc>] 0x000000008000e7cc
> [    1.869609] [<000000008000e80c>] 0x000000008000e80c
> [    1.874472] [<0000000080001e44>] 0x0000000080001e44
> [    1.879334] [<0000000080000a80>] 0x0000000080000a80
> [    1.884196] [<0000000080000ce4>] 0x0000000080000ce4
> [    1.889058] [<00000000801fd934>] 0x00000000801fd934
> [    1.893920] [<000000008011b304>] 0x000000008011b304
> [    1.899086] Mem-Info:
> [    1.901072] active_anon:0 inactive_anon:0 isolated_anon:0
> [    1.901072]  active_file:0 inactive_file:0 isolated_file:0
> [    1.901072]  unevictable:705 dirty:0 writeback:0 unstable:0
> [    1.901072]  slab_reclaimable:0 slab_unreclaimable:215
> [    1.901072]  mapped:0 shmem:0 pagetables:0 bounce:0
> [    1.901072]  free:417 free_pcp:0 free_cma:0
> [    1.931708] Node 0 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:2820kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
> [    1.953051] DMA32 free:1668kB min:4392kB low:4464kB high:4536kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:2820kB writepending:0kB present:8192kB managed:5528kB mlocked:0kB kernel_stack:168kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
> [    1.981067] lowmem_reserve[]: 0 0 0
> [    1.984492] DMA32: 1*4kB (U) 0*8kB 0*16kB 0*32kB 0*64kB 1*128kB (U) 0*256kB 1*512kB (U) 1*1024kB (U) 0*2048kB 0*4096kB = 1668kB
> [    1.995970] 704 total pagecache pages
> [    1.999599] 2048 pages RAM
> [    2.002291] 0 pages HighMem/MovableOnly
> [    2.006110] 666 pages reserved
> [    2.009131] Tasks state (memory values in pages):
> [    2.013848] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
> [    2.022450] Out of memory and no killable processes...
> [    2.027562] Kernel panic - not syncing: System is deadlocked on memory
> [    2.034062] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.6.0-rc1-00054-gd32122774dab #12
> [    2.042036] Call Trace:
> [    2.044472] [<000000008011c2f2>] 0x000000008011c2f2
> [    2.049333] [<000000008011c436>] 0x000000008011c436
> [    2.054195] [<00000000801ed86e>] 0x00000000801ed86e
> [    2.059057] [<000000008011f4d8>] 0x000000008011f4d8
> [    2.063919] [<00000000801633ea>] 0x00000000801633ea
> [    2.068782] [<000000008017451e>] 0x000000008017451e
> [    2.073644] [<00000000801746da>] 0x00000000801746da
> [    2.078506] [<0000000080161648>] 0x0000000080161648
> [    2.083368] [<000000008016241e>] 0x000000008016241e
> [    2.088230] [<0000000080192fc6>] 0x0000000080192fc6
> [    2.093092] [<00000000801624a6>] 0x00000000801624a6
> [    2.097954] [<000000008016262c>] 0x000000008016262c
> [    2.102816] [<000000008016267e>] 0x000000008016267e
> [    2.107678] [<0000000080178178>] 0x0000000080178178
> [    2.112541] [<00000000801781c0>] 0x00000000801781c0
> [    2.117403] [<0000000080178b5c>] 0x0000000080178b5c
> [    2.122265] [<0000000080178c82>] 0x0000000080178c82
> [    2.127127] [<0000000080001678>] 0x0000000080001678
> [    2.131989] [<0000000080001724>] 0x0000000080001724
> [    2.136851] [<000000008000145a>] 0x000000008000145a
> [    2.141713] [<00000000800014b0>] 0x00000000800014b0
> [    2.146575] [<000000008000e7cc>] 0x000000008000e7cc
> [    2.151437] [<000000008000e80c>] 0x000000008000e80c
> [    2.156299] [<0000000080001e44>] 0x0000000080001e44
> [    2.161162] [<0000000080000a80>] 0x0000000080000a80
> [    2.166024] [<0000000080000ce4>] 0x0000000080000ce4
> [    2.170886] [<00000000801fd934>] 0x00000000801fd934
> [    2.175748] [<000000008011b304>] 0x000000008011b304
> [    2.180624] ---[ end Kernel panic - not syncing: System is deadlocked on memory ]---
> 
> 
> 
>

Sean Anderson March 2, 2020, 3:53 a.m. UTC | #8

On 3/1/20 10:01 PM, Damien Le Moal wrote:
> On 2020/02/29 5:32, Sean Anderson wrote:
>> Hi,
>>
>> When booting from U-Boot I get an OOM. Perhaps this is related to the
>> second cpu not coming up?
> 
> Unlikely. It looks like your user space needs 2MB per shell (order 9
> allocation). Since you have only 5.5MB free, that may explain the allocation
> failure (if init is forking another shell especially).

This should be before init comes up; when comparing this to your syslog
output there are several more messages before init gets started.

> For the second core not coming up, an IPI needs to be sent to the non-boot core
> to wake it up. A Kendryte thing. U-boot should probably do it before jumping to
> the kernel. I thought I had that in the kernel though. Will check again.

I think it's a RISC-V thing. I should have U-Boot set up to start linux
on both cores, but something may be misconfigured on that end.

--Sean

Damien Le Moal March 2, 2020, 4:11 a.m. UTC | #9

On 2020/03/02 12:53, Sean Anderson wrote:
> On 3/1/20 10:01 PM, Damien Le Moal wrote:
>> On 2020/02/29 5:32, Sean Anderson wrote:
>>> Hi,
>>>
>>> When booting from U-Boot I get an OOM. Perhaps this is related to the
>>> second cpu not coming up?
>>
>> Unlikely. It looks like your user space needs 2MB per shell (order 9
>> allocation). Since you have only 5.5MB free, that may explain the allocation
>> failure (if init is forking another shell especially).
> 
> This should be before init comes up; when comparing this to your syslog
> output there are several more messages before init gets started.

Ah, yes. Your log shows:
[    1.899086] Mem-Info:
[    1.901072] active_anon:0 inactive_anon:0 isolated_anon:0
[    1.901072]  active_file:0 inactive_file:0 isolated_file:0
[    1.901072]  unevictable:705 dirty:0 writeback:0 unstable:0
[    1.901072]  slab_reclaimable:0 slab_unreclaimable:215
[    1.901072]  mapped:0 shmem:0 pagetables:0 bounce:0
[    1.901072]  free:417 free_pcp:0 free_cma:0

so only 417 free pages, but awapper is asking for 512 pages allocation... Weird.
Did you use the k210 default config ? Something using too much memory for the
board has been added to the config I think.

>> For the second core not coming up, an IPI needs to be sent to the non-boot core
>> to wake it up. A Kendryte thing. U-boot should probably do it before jumping to
>> the kernel. I thought I had that in the kernel though. Will check again.
> 
> I think it's a RISC-V thing. I should have U-Boot set up to start linux
> on both cores, but something may be misconfigured on that end.

May be. I would need to check the specs again :)
I think that normally, the rom boot stage is normally bringing up all cores ?
And the kendryte one does not ? Hence the need to explicitly do it for direct
boot (in the kernel) or in u-boot ?

> 
> --Sean
>

Anup Patel March 2, 2020, 4:17 a.m. UTC | #10

On Mon, Mar 2, 2020 at 9:23 AM Sean Anderson <seanga2@gmail.com> wrote:
>
> On 3/1/20 10:01 PM, Damien Le Moal wrote:
> > On 2020/02/29 5:32, Sean Anderson wrote:
> >> Hi,
> >>
> >> When booting from U-Boot I get an OOM. Perhaps this is related to the
> >> second cpu not coming up?
> >
> > Unlikely. It looks like your user space needs 2MB per shell (order 9
> > allocation). Since you have only 5.5MB free, that may explain the allocation
> > failure (if init is forking another shell especially).
>
> This should be before init comes up; when comparing this to your syslog
> output there are several more messages before init gets started.
>
> > For the second core not coming up, an IPI needs to be sent to the non-boot core
> > to wake it up. A Kendryte thing. U-boot should probably do it before jumping to
> > the kernel. I thought I had that in the kernel though. Will check again.
>
> I think it's a RISC-V thing. I should have U-Boot set up to start linux
> on both cores, but something may be misconfigured on that end.

You have to booti or bootm on U-Boot M-mode to make all CPUs jump to
Linux NOMMU.

Based on you log, it seems the second CPU is still spinning in U-Boot
M-mode and when Linux NOMMU tries to touch memory where second
CPU is spinning everything gets corrupted.

Regards,
Anup

>
> --Sean
>

Sean Anderson March 2, 2020, 4:18 a.m. UTC | #11

On 3/1/20 11:11 PM, Damien Le Moal wrote:
> On 2020/03/02 12:53, Sean Anderson wrote:
>> On 3/1/20 10:01 PM, Damien Le Moal wrote:
>>> On 2020/02/29 5:32, Sean Anderson wrote:
>>>> Hi,
>>>>
>>>> When booting from U-Boot I get an OOM. Perhaps this is related to the
>>>> second cpu not coming up?
>>>
>>> Unlikely. It looks like your user space needs 2MB per shell (order 9
>>> allocation). Since you have only 5.5MB free, that may explain the allocation
>>> failure (if init is forking another shell especially).
>>
>> This should be before init comes up; when comparing this to your syslog
>> output there are several more messages before init gets started.
> 
> Ah, yes. Your log shows:
> [    1.899086] Mem-Info:
> [    1.901072] active_anon:0 inactive_anon:0 isolated_anon:0
> [    1.901072]  active_file:0 inactive_file:0 isolated_file:0
> [    1.901072]  unevictable:705 dirty:0 writeback:0 unstable:0
> [    1.901072]  slab_reclaimable:0 slab_unreclaimable:215
> [    1.901072]  mapped:0 shmem:0 pagetables:0 bounce:0
> [    1.901072]  free:417 free_pcp:0 free_cma:0
> 
> so only 417 free pages, but awapper is asking for 512 pages allocation... Weird.
> Did you use the k210 default config ? Something using too much memory for the
> board has been added to the config I think.

I am using the default config. I thought it might be the initramfs
taking too much space on decompression, but I got the same problem when
using an uncompressed initramfs.

>>> For the second core not coming up, an IPI needs to be sent to the non-boot core
>>> to wake it up. A Kendryte thing. U-boot should probably do it before jumping to
>>> the kernel. I thought I had that in the kernel though. Will check again.
>>
>> I think it's a RISC-V thing. I should have U-Boot set up to start linux
>> on both cores, but something may be misconfigured on that end.
> 
> May be. I would need to check the specs again :)
> I think that normally, the rom boot stage is normally bringing up all cores ?
> And the kendryte one does not ? Hence the need to explicitly do it for direct
> boot (in the kernel) or in u-boot ?

The Kendryte rom brings up all the cores. However, that means that all
cores are executing the same code at the same time. So (in U-Boot) there
is a "hart lottery", which picks a hart to do booting from. All the
other harts instead jump to a loop where they wait for an IPI to tell
them what to execute. This same process also happened in the Kendryte
ROM, and will happen again when Linux comes up. The problem could be
related to my usage of the "go" command vs the full bootm process... I
will check to see if I have the same problem if I boot directly to
Linux.

--Sean

Sean Anderson March 2, 2020, 4:21 a.m. UTC | #12

On 3/1/20 11:17 PM, Anup Patel wrote:
> On Mon, Mar 2, 2020 at 9:23 AM Sean Anderson <seanga2@gmail.com> wrote:
>>
>> On 3/1/20 10:01 PM, Damien Le Moal wrote:
>>> On 2020/02/29 5:32, Sean Anderson wrote:
>>>> Hi,
>>>>
>>>> When booting from U-Boot I get an OOM. Perhaps this is related to the
>>>> second cpu not coming up?
>>>
>>> Unlikely. It looks like your user space needs 2MB per shell (order 9
>>> allocation). Since you have only 5.5MB free, that may explain the allocation
>>> failure (if init is forking another shell especially).
>>
>> This should be before init comes up; when comparing this to your syslog
>> output there are several more messages before init gets started.
>>
>>> For the second core not coming up, an IPI needs to be sent to the non-boot core
>>> to wake it up. A Kendryte thing. U-boot should probably do it before jumping to
>>> the kernel. I thought I had that in the kernel though. Will check again.
>>
>> I think it's a RISC-V thing. I should have U-Boot set up to start linux
>> on both cores, but something may be misconfigured on that end.
> 
> You have to booti or bootm on U-Boot M-mode to make all CPUs jump to
> Linux NOMMU.

Ah, I used just "go" for this test since bootm was having some problems
finding a place for the device tree. Perhaps I should try booting this
as a legacy standalone image...

> Based on you log, it seems the second CPU is still spinning in U-Boot
> M-mode and when Linux NOMMU tries to touch memory where second
> CPU is spinning everything gets corrupted.

--Sean

Damien Le Moal March 2, 2020, 4:48 a.m. UTC | #13

On 2020/03/02 13:17, Anup Patel wrote:
> On Mon, Mar 2, 2020 at 9:23 AM Sean Anderson <seanga2@gmail.com> wrote:
>>
>> On 3/1/20 10:01 PM, Damien Le Moal wrote:
>>> On 2020/02/29 5:32, Sean Anderson wrote:
>>>> Hi,
>>>>
>>>> When booting from U-Boot I get an OOM. Perhaps this is related to the
>>>> second cpu not coming up?
>>>
>>> Unlikely. It looks like your user space needs 2MB per shell (order 9
>>> allocation). Since you have only 5.5MB free, that may explain the allocation
>>> failure (if init is forking another shell especially).
>>
>> This should be before init comes up; when comparing this to your syslog
>> output there are several more messages before init gets started.
>>
>>> For the second core not coming up, an IPI needs to be sent to the non-boot core
>>> to wake it up. A Kendryte thing. U-boot should probably do it before jumping to
>>> the kernel. I thought I had that in the kernel though. Will check again.
>>
>> I think it's a RISC-V thing. I should have U-Boot set up to start linux
>> on both cores, but something may be misconfigured on that end.
> 
> You have to booti or bootm on U-Boot M-mode to make all CPUs jump to
> Linux NOMMU.
> 
> Based on you log, it seems the second CPU is still spinning in U-Boot
> M-mode and when Linux NOMMU tries to touch memory where second
> CPU is spinning everything gets corrupted.

Yes, I understand. But in the case of the K210, that last 2MB segment is really
special as by default it is not usable as regular SRAM. I think it may be better
to reflect that in the device tree. The K210 soc_early_init() call back can
probe for that special entry too to see if it has to be turned on for use as
regular memory or not (i.e. if a kpu driver will use it).

Since booting Linux with 6MB of memory will be even more challenging than with
8, I agree that we may never see the case of a kpu driver being used. But I
personally like making that special case clear in the device tree. No strong
objection to your simplification though. So if you really object, I will go with it.

Cheers.

Damien Le Moal March 2, 2020, 4:51 a.m. UTC | #14

On 2020/03/02 13:48, Damien Le Moal wrote:
> On 2020/03/02 13:17, Anup Patel wrote:
>> On Mon, Mar 2, 2020 at 9:23 AM Sean Anderson <seanga2@gmail.com> wrote:
>>>
>>> On 3/1/20 10:01 PM, Damien Le Moal wrote:
>>>> On 2020/02/29 5:32, Sean Anderson wrote:
>>>>> Hi,
>>>>>
>>>>> When booting from U-Boot I get an OOM. Perhaps this is related to the
>>>>> second cpu not coming up?
>>>>
>>>> Unlikely. It looks like your user space needs 2MB per shell (order 9
>>>> allocation). Since you have only 5.5MB free, that may explain the allocation
>>>> failure (if init is forking another shell especially).
>>>
>>> This should be before init comes up; when comparing this to your syslog
>>> output there are several more messages before init gets started.
>>>
>>>> For the second core not coming up, an IPI needs to be sent to the non-boot core
>>>> to wake it up. A Kendryte thing. U-boot should probably do it before jumping to
>>>> the kernel. I thought I had that in the kernel though. Will check again.
>>>
>>> I think it's a RISC-V thing. I should have U-Boot set up to start linux
>>> on both cores, but something may be misconfigured on that end.
>>
>> You have to booti or bootm on U-Boot M-mode to make all CPUs jump to
>> Linux NOMMU.
>>
>> Based on you log, it seems the second CPU is still spinning in U-Boot
>> M-mode and when Linux NOMMU tries to touch memory where second
>> CPU is spinning everything gets corrupted.
> 
> Yes, I understand. But in the case of the K210, that last 2MB segment is really
> special as by default it is not usable as regular SRAM. I think it may be better
> to reflect that in the device tree. The K210 soc_early_init() call back can
> probe for that special entry too to see if it has to be turned on for use as
> regular memory or not (i.e. if a kpu driver will use it).
> 
> Since booting Linux with 6MB of memory will be even more challenging than with
> 8, I agree that we may never see the case of a kpu driver being used. But I
> personally like making that special case clear in the device tree. No strong
> objection to your simplification though. So if you really object, I will go with it.

Answer to the wrong email... Please ignore :)

Damien Le Moal March 2, 2020, 4:54 a.m. UTC | #15

On 2020/03/02 13:18, Sean Anderson wrote:
> On 3/1/20 11:11 PM, Damien Le Moal wrote:
>> On 2020/03/02 12:53, Sean Anderson wrote:
>>> On 3/1/20 10:01 PM, Damien Le Moal wrote:
>>>> On 2020/02/29 5:32, Sean Anderson wrote:
>>>>> Hi,
>>>>>
>>>>> When booting from U-Boot I get an OOM. Perhaps this is related to the
>>>>> second cpu not coming up?
>>>>
>>>> Unlikely. It looks like your user space needs 2MB per shell (order 9
>>>> allocation). Since you have only 5.5MB free, that may explain the allocation
>>>> failure (if init is forking another shell especially).
>>>
>>> This should be before init comes up; when comparing this to your syslog
>>> output there are several more messages before init gets started.
>>
>> Ah, yes. Your log shows:
>> [    1.899086] Mem-Info:
>> [    1.901072] active_anon:0 inactive_anon:0 isolated_anon:0
>> [    1.901072]  active_file:0 inactive_file:0 isolated_file:0
>> [    1.901072]  unevictable:705 dirty:0 writeback:0 unstable:0
>> [    1.901072]  slab_reclaimable:0 slab_unreclaimable:215
>> [    1.901072]  mapped:0 shmem:0 pagetables:0 bounce:0
>> [    1.901072]  free:417 free_pcp:0 free_cma:0
>>
>> so only 417 free pages, but awapper is asking for 512 pages allocation... Weird.
>> Did you use the k210 default config ? Something using too much memory for the
>> board has been added to the config I think.
> 
> I am using the default config. I thought it might be the initramfs
> taking too much space on decompression, but I got the same problem when
> using an uncompressed initramfs.

Yes, that must be it. How big is your initramfs ? I generally do not see any
problem with 500-600KB initramfs. For bigger ones, I do see the OOM problem
often too. But most of the time, the OOM triggers more if the busybox executable
is too big and there are too many shell levels.

>>>> For the second core not coming up, an IPI needs to be sent to the non-boot core
>>>> to wake it up. A Kendryte thing. U-boot should probably do it before jumping to
>>>> the kernel. I thought I had that in the kernel though. Will check again.
>>>
>>> I think it's a RISC-V thing. I should have U-Boot set up to start linux
>>> on both cores, but something may be misconfigured on that end.
>>
>> May be. I would need to check the specs again :)
>> I think that normally, the rom boot stage is normally bringing up all cores ?
>> And the kendryte one does not ? Hence the need to explicitly do it for direct
>> boot (in the kernel) or in u-boot ?
> 
> The Kendryte rom brings up all the cores. However, that means that all
> cores are executing the same code at the same time. So (in U-Boot) there
> is a "hart lottery", which picks a hart to do booting from. All the
> other harts instead jump to a loop where they wait for an IPI to tell
> them what to execute. This same process also happened in the Kendryte
> ROM, and will happen again when Linux comes up. The problem could be
> related to my usage of the "go" command vs the full bootm process... I
> will check to see if I have the same problem if I boot directly to
> Linux.
> 
> --Sean
> 
>

Sean Anderson March 2, 2020, 4:56 a.m. UTC | #16

On 3/1/20 11:54 PM, Damien Le Moal wrote:
> On 2020/03/02 13:18, Sean Anderson wrote:
>> On 3/1/20 11:11 PM, Damien Le Moal wrote:
>>> On 2020/03/02 12:53, Sean Anderson wrote:
>>>> On 3/1/20 10:01 PM, Damien Le Moal wrote:
>>>>> On 2020/02/29 5:32, Sean Anderson wrote:
>>>>>> Hi,
>>>>>>
>>>>>> When booting from U-Boot I get an OOM. Perhaps this is related to the
>>>>>> second cpu not coming up?
>>>>>
>>>>> Unlikely. It looks like your user space needs 2MB per shell (order 9
>>>>> allocation). Since you have only 5.5MB free, that may explain the allocation
>>>>> failure (if init is forking another shell especially).
>>>>
>>>> This should be before init comes up; when comparing this to your syslog
>>>> output there are several more messages before init gets started.
>>>
>>> Ah, yes. Your log shows:
>>> [    1.899086] Mem-Info:
>>> [    1.901072] active_anon:0 inactive_anon:0 isolated_anon:0
>>> [    1.901072]  active_file:0 inactive_file:0 isolated_file:0
>>> [    1.901072]  unevictable:705 dirty:0 writeback:0 unstable:0
>>> [    1.901072]  slab_reclaimable:0 slab_unreclaimable:215
>>> [    1.901072]  mapped:0 shmem:0 pagetables:0 bounce:0
>>> [    1.901072]  free:417 free_pcp:0 free_cma:0
>>>
>>> so only 417 free pages, but awapper is asking for 512 pages allocation... Weird.
>>> Did you use the k210 default config ? Something using too much memory for the
>>> board has been added to the config I think.
>>
>> I am using the default config. I thought it might be the initramfs
>> taking too much space on decompression, but I got the same problem when
>> using an uncompressed initramfs.
> 
> Yes, that must be it. How big is your initramfs ? I generally do not see any
> problem with 500-600KB initramfs. For bigger ones, I do see the OOM problem
> often too. But most of the time, the OOM triggers more if the busybox executable
> is too big and there are too many shell levels.

My initramfs is 1.6M atm, so perhaps I should try with a smaller one.

--Sean

Sean Anderson March 2, 2020, 5:02 a.m. UTC | #17

On 3/1/20 11:48 PM, Damien Le Moal wrote:
> On 2020/03/02 13:17, Anup Patel wrote:
>> On Mon, Mar 2, 2020 at 9:23 AM Sean Anderson <seanga2@gmail.com> wrote:
>>>
>>> On 3/1/20 10:01 PM, Damien Le Moal wrote:
>>>> On 2020/02/29 5:32, Sean Anderson wrote:
>>>>> Hi,
>>>>>
>>>>> When booting from U-Boot I get an OOM. Perhaps this is related to the
>>>>> second cpu not coming up?
>>>>
>>>> Unlikely. It looks like your user space needs 2MB per shell (order 9
>>>> allocation). Since you have only 5.5MB free, that may explain the allocation
>>>> failure (if init is forking another shell especially).
>>>
>>> This should be before init comes up; when comparing this to your syslog
>>> output there are several more messages before init gets started.
>>>
>>>> For the second core not coming up, an IPI needs to be sent to the non-boot core
>>>> to wake it up. A Kendryte thing. U-boot should probably do it before jumping to
>>>> the kernel. I thought I had that in the kernel though. Will check again.
>>>
>>> I think it's a RISC-V thing. I should have U-Boot set up to start linux
>>> on both cores, but something may be misconfigured on that end.
>>
>> You have to booti or bootm on U-Boot M-mode to make all CPUs jump to
>> Linux NOMMU.
>>
>> Based on you log, it seems the second CPU is still spinning in U-Boot
>> M-mode and when Linux NOMMU tries to touch memory where second
>> CPU is spinning everything gets corrupted.
> 
> Yes, I understand. But in the case of the K210, that last 2MB segment is really
> special as by default it is not usable as regular SRAM. I think it may be better
> to reflect that in the device tree. The K210 soc_early_init() call back can
> probe for that special entry too to see if it has to be turned on for use as
> regular memory or not (i.e. if a kpu driver will use it).
> 
> Since booting Linux with 6MB of memory will be even more challenging than with
> 8, I agree that we may never see the case of a kpu driver being used. But I
> personally like making that special case clear in the device tree. No strong
> objection to your simplification though. So if you really object, I will go with it.

The way I have it set up is like this


	sram0: memory@80000000 {
		device_type = "memory";
		compatible = "kendryte,k210-sram";
		reg = <0x80000000 0x400000>;
		clocks = <&sysclk K210_CLK_SRAM0>;
	};

	sram1: memory@80400000 {
		device_type = "memory";
		reg = <0x80400000 0x200000>;
		compatible = "kendryte,k210-sram";
		clocks = <&sysclk K210_CLK_SRAM1>;
	};

	airam: memory@80600000 {
		device_type = "memory";
		reg = <0x80600000 0x200000>;
		compatible = "kendryte,k210-airam";
		clocks = <&sysclk K210_CLK_AI>;
	};

	reserved-memory {
		#address-cells = <1>;
		#size-cells = <1>;
		ranges;

		ai_reserved: ai@80600000 {
			reg = <0x80600000 0x200000>;
			reusable;
		};
	};

And then the kpu has 'memory-region = <&ai_reserved>;'. This way the
memory is documented as being used by the kpu, but also free when the
KPU is not in use.

However, I have been unable to get the AI ram to work; any time I
access it the CPU hangs. I have tried all combinations of

* Enabling/disabling the AI clock
* Enabling/disabling PLL1 (the AI clock's parent) but not the AI clock
* Asserting/deasserting the KPU reset

but I have not been able to get it working. Have you had any luck?

There's also the issue that all DMAs should happen from the uncached
memory area, but I think there is some existing infrastructure to
"translate" IO addresses, so this doesn't need to be added to the device
tree.

--Sean

Damien Le Moal March 2, 2020, 5:03 a.m. UTC | #18

On 2020/03/02 13:57, Sean Anderson wrote:
> On 3/1/20 11:54 PM, Damien Le Moal wrote:
>> On 2020/03/02 13:18, Sean Anderson wrote:
>>> On 3/1/20 11:11 PM, Damien Le Moal wrote:
>>>> On 2020/03/02 12:53, Sean Anderson wrote:
>>>>> On 3/1/20 10:01 PM, Damien Le Moal wrote:
>>>>>> On 2020/02/29 5:32, Sean Anderson wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> When booting from U-Boot I get an OOM. Perhaps this is related to the
>>>>>>> second cpu not coming up?
>>>>>>
>>>>>> Unlikely. It looks like your user space needs 2MB per shell (order 9
>>>>>> allocation). Since you have only 5.5MB free, that may explain the allocation
>>>>>> failure (if init is forking another shell especially).
>>>>>
>>>>> This should be before init comes up; when comparing this to your syslog
>>>>> output there are several more messages before init gets started.
>>>>
>>>> Ah, yes. Your log shows:
>>>> [    1.899086] Mem-Info:
>>>> [    1.901072] active_anon:0 inactive_anon:0 isolated_anon:0
>>>> [    1.901072]  active_file:0 inactive_file:0 isolated_file:0
>>>> [    1.901072]  unevictable:705 dirty:0 writeback:0 unstable:0
>>>> [    1.901072]  slab_reclaimable:0 slab_unreclaimable:215
>>>> [    1.901072]  mapped:0 shmem:0 pagetables:0 bounce:0
>>>> [    1.901072]  free:417 free_pcp:0 free_cma:0
>>>>
>>>> so only 417 free pages, but awapper is asking for 512 pages allocation... Weird.
>>>> Did you use the k210 default config ? Something using too much memory for the
>>>> board has been added to the config I think.
>>>
>>> I am using the default config. I thought it might be the initramfs
>>> taking too much space on decompression, but I got the same problem when
>>> using an uncompressed initramfs.
>>
>> Yes, that must be it. How big is your initramfs ? I generally do not see any
>> problem with 500-600KB initramfs. For bigger ones, I do see the OOM problem
>> often too. But most of the time, the OOM triggers more if the busybox executable
>> is too big and there are too many shell levels.
> 
> My initramfs is 1.6M atm, so perhaps I should try with a smaller one.

Most likely. I never had success initially with such big initramfs. Buildroot
initramfs builds tend to be really much bigger than necessary. So I tend to
manually make my own. With a 1.6MB initramfs, you will need the order 9
allocation seen (2MB) before the 1.6MB embedded initramfs is freed. With some
memory fragmentation, that may fail easily. I do wonder though why the memory
would be so fragmented so early in the boot process. It may be interesting to
explore that to see if some optimizations cannot be added.

Damien Le Moal March 2, 2020, 5:11 a.m. UTC | #19

On 2020/03/02 14:02, Sean Anderson wrote:
> On 3/1/20 11:48 PM, Damien Le Moal wrote:
>> On 2020/03/02 13:17, Anup Patel wrote:
>>> On Mon, Mar 2, 2020 at 9:23 AM Sean Anderson <seanga2@gmail.com> wrote:
>>>>
>>>> On 3/1/20 10:01 PM, Damien Le Moal wrote:
>>>>> On 2020/02/29 5:32, Sean Anderson wrote:
>>>>>> Hi,
>>>>>>
>>>>>> When booting from U-Boot I get an OOM. Perhaps this is related to the
>>>>>> second cpu not coming up?
>>>>>
>>>>> Unlikely. It looks like your user space needs 2MB per shell (order 9
>>>>> allocation). Since you have only 5.5MB free, that may explain the allocation
>>>>> failure (if init is forking another shell especially).
>>>>
>>>> This should be before init comes up; when comparing this to your syslog
>>>> output there are several more messages before init gets started.
>>>>
>>>>> For the second core not coming up, an IPI needs to be sent to the non-boot core
>>>>> to wake it up. A Kendryte thing. U-boot should probably do it before jumping to
>>>>> the kernel. I thought I had that in the kernel though. Will check again.
>>>>
>>>> I think it's a RISC-V thing. I should have U-Boot set up to start linux
>>>> on both cores, but something may be misconfigured on that end.
>>>
>>> You have to booti or bootm on U-Boot M-mode to make all CPUs jump to
>>> Linux NOMMU.
>>>
>>> Based on you log, it seems the second CPU is still spinning in U-Boot
>>> M-mode and when Linux NOMMU tries to touch memory where second
>>> CPU is spinning everything gets corrupted.
>>
>> Yes, I understand. But in the case of the K210, that last 2MB segment is really
>> special as by default it is not usable as regular SRAM. I think it may be better
>> to reflect that in the device tree. The K210 soc_early_init() call back can
>> probe for that special entry too to see if it has to be turned on for use as
>> regular memory or not (i.e. if a kpu driver will use it).
>>
>> Since booting Linux with 6MB of memory will be even more challenging than with
>> 8, I agree that we may never see the case of a kpu driver being used. But I
>> personally like making that special case clear in the device tree. No strong
>> objection to your simplification though. So if you really object, I will go with it.
> 
> The way I have it set up is like this
> 
> 
> 	sram0: memory@80000000 {
> 		device_type = "memory";
> 		compatible = "kendryte,k210-sram";
> 		reg = <0x80000000 0x400000>;
> 		clocks = <&sysclk K210_CLK_SRAM0>;
> 	};
> 
> 	sram1: memory@80400000 {
> 		device_type = "memory";
> 		reg = <0x80400000 0x200000>;
> 		compatible = "kendryte,k210-sram";
> 		clocks = <&sysclk K210_CLK_SRAM1>;
> 	};
> 
> 	airam: memory@80600000 {
> 		device_type = "memory";
> 		reg = <0x80600000 0x200000>;
> 		compatible = "kendryte,k210-airam";
> 		clocks = <&sysclk K210_CLK_AI>;
> 	};
> 
> 	reserved-memory {
> 		#address-cells = <1>;
> 		#size-cells = <1>;
> 		ranges;
> 
> 		ai_reserved: ai@80600000 {
> 			reg = <0x80600000 0x200000>;
> 			reusable;
> 		};
> 	};>
> And then the kpu has 'memory-region = <&ai_reserved>;'. This way the
> memory is documented as being used by the kpu, but also free when the
> KPU is not in use.

I tried to use your syntax above initially but (if I remember correctly), the
"reserved-memory" entry prevents the initial ram setup to add the kpu segment,
so you can never see it as regular memory. So I dropped that and that memory is
usable with the v1 of the series I sent. The soc_early_init() enables it by
turning on pll1.

> 
> However, I have been unable to get the AI ram to work; any time I
> access it the CPU hangs. I have tried all combinations of
> 
> * Enabling/disabling the AI clock
> * Enabling/disabling PLL1 (the AI clock's parent) but not the AI clock
> * Asserting/deasserting the KPU reset
> 
> but I have not been able to get it working. Have you had any luck?

See k210_soc_early_init() in drivers/soc/kendryte/k210-sysctl.c. That works.
You did already point out the clock initialization that is not enough and
working only because most of the values are the default. Maybe U-boot is
changing some of them resulting in the worng clock frequencies being set in the
kernel ?

> 
> There's also the issue that all DMAs should happen from the uncached
> memory area, but I think there is some existing infrastructure to
> "translate" IO addresses, so this doesn't need to be added to the device
> tree.
> 
> --Sean
> 
>

Sean Anderson March 2, 2020, 5:25 a.m. UTC | #20

On 3/2/20 12:11 AM, Damien Le Moal wrote:
> On 2020/03/02 14:02, Sean Anderson wrote:
>> On 3/1/20 11:48 PM, Damien Le Moal wrote:
>>> On 2020/03/02 13:17, Anup Patel wrote:
>>>> On Mon, Mar 2, 2020 at 9:23 AM Sean Anderson <seanga2@gmail.com> wrote:
>>>>>
>>>>> On 3/1/20 10:01 PM, Damien Le Moal wrote:
>>>>>> On 2020/02/29 5:32, Sean Anderson wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> When booting from U-Boot I get an OOM. Perhaps this is related to the
>>>>>>> second cpu not coming up?
>>>>>>
>>>>>> Unlikely. It looks like your user space needs 2MB per shell (order 9
>>>>>> allocation). Since you have only 5.5MB free, that may explain the allocation
>>>>>> failure (if init is forking another shell especially).
>>>>>
>>>>> This should be before init comes up; when comparing this to your syslog
>>>>> output there are several more messages before init gets started.
>>>>>
>>>>>> For the second core not coming up, an IPI needs to be sent to the non-boot core
>>>>>> to wake it up. A Kendryte thing. U-boot should probably do it before jumping to
>>>>>> the kernel. I thought I had that in the kernel though. Will check again.
>>>>>
>>>>> I think it's a RISC-V thing. I should have U-Boot set up to start linux
>>>>> on both cores, but something may be misconfigured on that end.
>>>>
>>>> You have to booti or bootm on U-Boot M-mode to make all CPUs jump to
>>>> Linux NOMMU.
>>>>
>>>> Based on you log, it seems the second CPU is still spinning in U-Boot
>>>> M-mode and when Linux NOMMU tries to touch memory where second
>>>> CPU is spinning everything gets corrupted.
>>>
>>> Yes, I understand. But in the case of the K210, that last 2MB segment is really
>>> special as by default it is not usable as regular SRAM. I think it may be better
>>> to reflect that in the device tree. The K210 soc_early_init() call back can
>>> probe for that special entry too to see if it has to be turned on for use as
>>> regular memory or not (i.e. if a kpu driver will use it).
>>>
>>> Since booting Linux with 6MB of memory will be even more challenging than with
>>> 8, I agree that we may never see the case of a kpu driver being used. But I
>>> personally like making that special case clear in the device tree. No strong
>>> objection to your simplification though. So if you really object, I will go with it.
>>
>> The way I have it set up is like this
>>
>>
>> 	sram0: memory@80000000 {
>> 		device_type = "memory";
>> 		compatible = "kendryte,k210-sram";
>> 		reg = <0x80000000 0x400000>;
>> 		clocks = <&sysclk K210_CLK_SRAM0>;
>> 	};
>>
>> 	sram1: memory@80400000 {
>> 		device_type = "memory";
>> 		reg = <0x80400000 0x200000>;
>> 		compatible = "kendryte,k210-sram";
>> 		clocks = <&sysclk K210_CLK_SRAM1>;
>> 	};
>>
>> 	airam: memory@80600000 {
>> 		device_type = "memory";
>> 		reg = <0x80600000 0x200000>;
>> 		compatible = "kendryte,k210-airam";
>> 		clocks = <&sysclk K210_CLK_AI>;
>> 	};
>>
>> 	reserved-memory {
>> 		#address-cells = <1>;
>> 		#size-cells = <1>;
>> 		ranges;
>>
>> 		ai_reserved: ai@80600000 {
>> 			reg = <0x80600000 0x200000>;
>> 			reusable;
>> 		};
>> 	};>
>> And then the kpu has 'memory-region = <&ai_reserved>;'. This way the
>> memory is documented as being used by the kpu, but also free when the
>> KPU is not in use.
> 
> I tried to use your syntax above initially but (if I remember correctly), the
> "reserved-memory" entry prevents the initial ram setup to add the kpu segment,
> so you can never see it as regular memory. So I dropped that and that memory is
> usable with the v1 of the series I sent. The soc_early_init() enables it by
> turning on pll1.

This seems like it could be fixed by writing a dummy driver for the KPU
which does nothing but release the memory region.

> 
>>
>> However, I have been unable to get the AI ram to work; any time I
>> access it the CPU hangs. I have tried all combinations of
>>
>> * Enabling/disabling the AI clock
>> * Enabling/disabling PLL1 (the AI clock's parent) but not the AI clock
>> * Asserting/deasserting the KPU reset
>>
>> but I have not been able to get it working. Have you had any luck?
> 
> See k210_soc_early_init() in drivers/soc/kendryte/k210-sysctl.c. That works.
> You did already point out the clock initialization that is not enough and
> working only because most of the values are the default. Maybe U-boot is
> changing some of them resulting in the worng clock frequencies being set in the
> kernel ?

My current clock setup when booted looks like

=> clk dump 
 Rate               Id        Usecnt      Name
----------------------------------------------------
 26000000             0         2        |-- osc
 780000000            1         1        |   |-- pll0
 390000000            0         2        |   |   `-- pll0_half
 390000000           42         6        |   |       |-- aclk
 390000000            5         1        |   |       |   |-- sram0
 390000000            6         1        |   |       |   |-- sram1
 195000000           10         0        |   |       |   |-- rom
 390000000           13         0        |   |       |   |-- dvp
 195000000            7         2        |   |       |   |-- apb0
 195000000           15         0        |   |       |   |   |-- gpio
 195000000           29         0        |   |       |   |   |-- uart1
 195000000           30         0        |   |       |   |   |-- uart2
 195000000           31         0        |   |       |   |   |-- uart3
 195000000           33         1        |   |       |   |   |-- fpioa
 195000000           39         0        |   |       |   |   `-- sha
 195000000            8         1        |   |       |   |-- apb1
 195000000           32         0        |   |       |   |   |-- aes
 195000000           40         0        |   |       |   |   `-- otp
 195000000            9         1        |   |       |   |-- apb2
 390000000            4         2        |   |       |   |-- cpu
 390000000           11         0        |   |       |   |-- dma
 390000000           14         0        |   |       |   `-- fft
 97500000            19         0        |   |       |-- spi3
 390000000           34         0        |   |       |-- timer0
 390000000           35         0        |   |       |-- timer1
 390000000           36         0        |   |       |-- timer2
 390000000           16         0        |   |       |-- spi0
 390000000           17         1        |   |       |-- spi1
 390000000           18         0        |   |       |-- spi2
 390000000           26         0        |   |       |-- i2c0
 390000000           27         0        |   |       |-- i2c1
 390000000           28         0        |   |       `-- i2c2
 299000000            2         1        |   |-- pll1
 299000000           12         1        |   |   `-- ai
 0                    3         0        |   |-- pll2
 0                    0         0        |   |   `-- pll2_half
 0                   20         0        |   |       |-- i2s0
 0                   21         0        |   |       |-- i2s1
 0                   22         0        |   |       |-- i2s2
 0                   23         0        |   |       |-- i2s0_m
 0                   24         0        |   |       |-- i2s1_m
 0                   25         0        |   |       `-- i2s2_m
 13000000             0         0        |   |-- in0_half
 13000000            37         0        |   |   |-- wdt0
 13000000            38         0        |   |   `-- wdt1
 26000000            41         0        |   `-- rtc

Perhaps I need PLL1 enabled but *not* AI. Alternatively, I have PLL1 set
to the default rate of 299 MHz. It could be a clock domain problem.

>>
>> There's also the issue that all DMAs should happen from the uncached
>> memory area, but I think there is some existing infrastructure to
>> "translate" IO addresses, so this doesn't need to be added to the device
>> tree.
>>
>> --Sean
>>
>>
> 
>

Damien Le Moal March 2, 2020, 5:43 a.m. UTC | #21

On 2020/03/02 14:25, Sean Anderson wrote:
> On 3/2/20 12:11 AM, Damien Le Moal wrote:
>> On 2020/03/02 14:02, Sean Anderson wrote:
>>> On 3/1/20 11:48 PM, Damien Le Moal wrote:
>>>> On 2020/03/02 13:17, Anup Patel wrote:
>>>>> On Mon, Mar 2, 2020 at 9:23 AM Sean Anderson <seanga2@gmail.com> wrote:
>>>>>>
>>>>>> On 3/1/20 10:01 PM, Damien Le Moal wrote:
>>>>>>> On 2020/02/29 5:32, Sean Anderson wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> When booting from U-Boot I get an OOM. Perhaps this is related to the
>>>>>>>> second cpu not coming up?
>>>>>>>
>>>>>>> Unlikely. It looks like your user space needs 2MB per shell (order 9
>>>>>>> allocation). Since you have only 5.5MB free, that may explain the allocation
>>>>>>> failure (if init is forking another shell especially).
>>>>>>
>>>>>> This should be before init comes up; when comparing this to your syslog
>>>>>> output there are several more messages before init gets started.
>>>>>>
>>>>>>> For the second core not coming up, an IPI needs to be sent to the non-boot core
>>>>>>> to wake it up. A Kendryte thing. U-boot should probably do it before jumping to
>>>>>>> the kernel. I thought I had that in the kernel though. Will check again.
>>>>>>
>>>>>> I think it's a RISC-V thing. I should have U-Boot set up to start linux
>>>>>> on both cores, but something may be misconfigured on that end.
>>>>>
>>>>> You have to booti or bootm on U-Boot M-mode to make all CPUs jump to
>>>>> Linux NOMMU.
>>>>>
>>>>> Based on you log, it seems the second CPU is still spinning in U-Boot
>>>>> M-mode and when Linux NOMMU tries to touch memory where second
>>>>> CPU is spinning everything gets corrupted.
>>>>
>>>> Yes, I understand. But in the case of the K210, that last 2MB segment is really
>>>> special as by default it is not usable as regular SRAM. I think it may be better
>>>> to reflect that in the device tree. The K210 soc_early_init() call back can
>>>> probe for that special entry too to see if it has to be turned on for use as
>>>> regular memory or not (i.e. if a kpu driver will use it).
>>>>
>>>> Since booting Linux with 6MB of memory will be even more challenging than with
>>>> 8, I agree that we may never see the case of a kpu driver being used. But I
>>>> personally like making that special case clear in the device tree. No strong
>>>> objection to your simplification though. So if you really object, I will go with it.
>>>
>>> The way I have it set up is like this
>>>
>>>
>>> 	sram0: memory@80000000 {
>>> 		device_type = "memory";
>>> 		compatible = "kendryte,k210-sram";
>>> 		reg = <0x80000000 0x400000>;
>>> 		clocks = <&sysclk K210_CLK_SRAM0>;
>>> 	};
>>>
>>> 	sram1: memory@80400000 {
>>> 		device_type = "memory";
>>> 		reg = <0x80400000 0x200000>;
>>> 		compatible = "kendryte,k210-sram";
>>> 		clocks = <&sysclk K210_CLK_SRAM1>;
>>> 	};
>>>
>>> 	airam: memory@80600000 {
>>> 		device_type = "memory";
>>> 		reg = <0x80600000 0x200000>;
>>> 		compatible = "kendryte,k210-airam";
>>> 		clocks = <&sysclk K210_CLK_AI>;
>>> 	};
>>>
>>> 	reserved-memory {
>>> 		#address-cells = <1>;
>>> 		#size-cells = <1>;
>>> 		ranges;
>>>
>>> 		ai_reserved: ai@80600000 {
>>> 			reg = <0x80600000 0x200000>;
>>> 			reusable;
>>> 		};
>>> 	};>
>>> And then the kpu has 'memory-region = <&ai_reserved>;'. This way the
>>> memory is documented as being used by the kpu, but also free when the
>>> KPU is not in use.
>>
>> I tried to use your syntax above initially but (if I remember correctly), the
>> "reserved-memory" entry prevents the initial ram setup to add the kpu segment,
>> so you can never see it as regular memory. So I dropped that and that memory is
>> usable with the v1 of the series I sent. The soc_early_init() enables it by
>> turning on pll1.
> 
> This seems like it could be fixed by writing a dummy driver for the KPU
> which does nothing but release the memory region.
> 
>>
>>>
>>> However, I have been unable to get the AI ram to work; any time I
>>> access it the CPU hangs. I have tried all combinations of
>>>
>>> * Enabling/disabling the AI clock
>>> * Enabling/disabling PLL1 (the AI clock's parent) but not the AI clock
>>> * Asserting/deasserting the KPU reset
>>>
>>> but I have not been able to get it working. Have you had any luck?
>>
>> See k210_soc_early_init() in drivers/soc/kendryte/k210-sysctl.c. That works.
>> You did already point out the clock initialization that is not enough and
>> working only because most of the values are the default. Maybe U-boot is
>> changing some of them resulting in the worng clock frequencies being set in the
>> kernel ?
> 
> My current clock setup when booted looks like
> 
> => clk dump 
>  Rate               Id        Usecnt      Name
> ----------------------------------------------------
>  26000000             0         2        |-- osc
>  780000000            1         1        |   |-- pll0
>  390000000            0         2        |   |   `-- pll0_half
>  390000000           42         6        |   |       |-- aclk
>  390000000            5         1        |   |       |   |-- sram0
>  390000000            6         1        |   |       |   |-- sram1
>  195000000           10         0        |   |       |   |-- rom
>  390000000           13         0        |   |       |   |-- dvp
>  195000000            7         2        |   |       |   |-- apb0
>  195000000           15         0        |   |       |   |   |-- gpio
>  195000000           29         0        |   |       |   |   |-- uart1
>  195000000           30         0        |   |       |   |   |-- uart2
>  195000000           31         0        |   |       |   |   |-- uart3
>  195000000           33         1        |   |       |   |   |-- fpioa
>  195000000           39         0        |   |       |   |   `-- sha
>  195000000            8         1        |   |       |   |-- apb1
>  195000000           32         0        |   |       |   |   |-- aes
>  195000000           40         0        |   |       |   |   `-- otp
>  195000000            9         1        |   |       |   |-- apb2
>  390000000            4         2        |   |       |   |-- cpu
>  390000000           11         0        |   |       |   |-- dma
>  390000000           14         0        |   |       |   `-- fft
>  97500000            19         0        |   |       |-- spi3
>  390000000           34         0        |   |       |-- timer0
>  390000000           35         0        |   |       |-- timer1
>  390000000           36         0        |   |       |-- timer2
>  390000000           16         0        |   |       |-- spi0
>  390000000           17         1        |   |       |-- spi1
>  390000000           18         0        |   |       |-- spi2
>  390000000           26         0        |   |       |-- i2c0
>  390000000           27         0        |   |       |-- i2c1
>  390000000           28         0        |   |       `-- i2c2
>  299000000            2         1        |   |-- pll1
>  299000000           12         1        |   |   `-- ai
>  0                    3         0        |   |-- pll2
>  0                    0         0        |   |   `-- pll2_half
>  0                   20         0        |   |       |-- i2s0
>  0                   21         0        |   |       |-- i2s1
>  0                   22         0        |   |       |-- i2s2
>  0                   23         0        |   |       |-- i2s0_m
>  0                   24         0        |   |       |-- i2s1_m
>  0                   25         0        |   |       `-- i2s2_m
>  13000000             0         0        |   |-- in0_half
>  13000000            37         0        |   |   |-- wdt0
>  13000000            38         0        |   |   `-- wdt1
>  26000000            41         0        |   `-- rtc
> 
> Perhaps I need PLL1 enabled but *not* AI. Alternatively, I have PLL1 set
> to the default rate of 299 MHz. It could be a clock domain problem.

I kind of remember reading that if you enable the AI clock, then the SRAM is not
usable as regular SRAM anymore and becomes reserved for the KPU. You need to
enable pll1 only for using it as regular mem. However, you mentioned that you
tried that already... Not sure what is going on.

Palmer Dabbelt March 4, 2020, 7:44 p.m. UTC | #22

On Wed, 12 Feb 2020 02:34:22 PST (-0800), Damien Le Moal wrote:
> This series adds support to boot nommu Linux on Kendryte K210 SoC based
> boards. This is all based on initial work done by Christoph Hellwig.
>
> The first 2 patches fix riscv gitignore and a potential nommu
> compilation error. These patches are not specific to the Kendryte
> support.

Thanks, I put #2 on fixes (after having already put #1 there earlier).

>
> Patch 3 adds unaligned load/store trap handlers for M-mode.
>
> Patch 4 enables a builtin DTB to allow passing a device tree to the
> kernel when the board bootchain is enabled to pass one.

I had some comments on that one, but otherwise this looks good.  LMK if you
want to re-spin this or if you'd like me to deal with it.

Thanks!

>
> Patch 5 introduces an early SoC initialization enabling very early
> hardware initialization not possible with device tree entries pointing
> to drivers. This is used in patch 6 which introduces a sysctl driver for
> the K210 SoC. The early SoC initialization is used to enable the
> additional 2MB of SRAM normally reserved to the SoC AI chip.
>
> Patch 7 to 9 add necessary Kconfig changes, a defconfig and a generic
> device tree suitable for many K210 boards.
>
> Finally, patch 10 adds compilation of a bootable image file (bin file)
> that can be used to flash the board ROM.
>
> This series was tested on the Kendryte KD233 development board, the
> Sipeed MAIX dan dock board and the Sipeed MAIXDUINO board. The userspace
> used was built using a modified buildroot tree for the toolchain part
> and an unmodified busybox tree for the initramfs image (embedded in the
> kernel as a cpio file). The folowwing github project contains the
> modified buildroot tree:
>
> https://github.com/damien-lemoal/riscv64-nommu-buildroot
>
> This is based on work from Christoph Hellwig, with additional changes
> and updates to the latest upstream versions for buildroot and uClibc.
>
> Precompiled versions of the toolchain (gcc 9.2) and initramfs file tree
> and cpio file can be found in this project under the directory:
>
> buildroot/riscv64-uclibc-nommu/
>
> Flashing the file arch/riscv/boot/loader.bin to a board can be done
> using the Sipeed kflash.py tool with the command:
>
> kflash.py/kflash.py -p /dev/ttyUSB0 -b 1500000 -t loader.bin
>
> The kflash.py tool can be found here:
>
> https://github.com/sipeed/kflash.py
>
> For reference, using the Sipeed MAIXDUINO board, here is the boot
> output:
>
> [INFO] Rebooting...
> --- forcing DTR inactive
> --- forcing RTS inactive
> --- Miniterm on /dev/ttyUSB0  115200,8,N,1 ---
> --- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
> [    0.000000] Linux version 5.6.0-rc1-00011-ga2b5be1c4374 (damien@yyy) (gcc version 8.2.0 (Buildroot 2018.11-rc2-00003-ga0787e9)) #610 SMP Wed Feb 12 18:53:50 JST 2020
> [    0.000000] earlycon: sifive0 at MMIO 0x0000000038000000 (options '')
> [    0.000000] printk: bootconsole [sifive0] enabled
> [    0.000000] initrd not found or empty - disabling initrd
> [    0.000000] Zone ranges:
> [    0.000000]   DMA32    [mem 0x0000000080000000-0x00000000807fffff]
> [    0.000000]   Normal   empty
> [    0.000000] Movable zone start for each node
> [    0.000000] Early memory node ranges
> [    0.000000]   node   0: [mem 0x0000000080000000-0x00000000807fffff]
> [    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x00000000807fffff]
> [    0.000000] elf_hwcap is 0x112d
> [    0.000000] percpu: max_distance=0x18000 too large for vmalloc space 0x0
> [    0.000000] percpu: Embedded 12 pages/cpu s18272 r0 d30880 u49152
> [    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2020
> [    0.000000] Kernel command line: earlycon console=ttySIF0
> [    0.000000] Dentry cache hash table entries: 1024 (order: 1, 8192 bytes, linear)
> [    0.000000] Inode-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
> [    0.000000] Sorting __ex_table...
> [    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
> [    0.000000] Memory: 6328K/8192K available (924K kernel code, 110K rwdata, 164K rodata, 321K init, 91K bss, 1864K reserved, 0K cma-reserved)
> [    0.000000] rcu: Hierarchical RCU implementation.
> [    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
> [    0.000000] NR_IRQS: 0, nr_irqs: 0, preallocated irqs: 0
> [    0.000000] plic: mapped 65 interrupts with 2 handlers for 4 contexts.
> [    0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
> [    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x3990be68b, max_idle_ns: 881590404272 ns
> [    0.000014] sched_clock: 64 bits at 7MHz, resolution 128ns, wraps every 4398046511054ns
> [    0.008232] Console: colour dummy device 80x25
> [    0.012474] Calibrating delay loop (skipped), value calculated using timer frequency.. 15.60 BogoMIPS (lpj=31200)
> [    0.022678] pid_max: default: 4096 minimum: 301
> [    0.027288] Mount-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
> [    0.034414] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
> [    0.044796] rcu: Hierarchical SRCU implementation.
> [    0.049602] smp: Bringing up secondary CPUs ...
> [    0.054746] smp: Brought up 1 node, 2 CPUs
> [    0.059093] devtmpfs: initialized
> [    0.065523] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
> [    0.074544] futex hash table entries: 16 (order: -2, 1024 bytes, linear)
> [    0.082512] Kendryte K210 SoC sysctl
> [    0.096010] clocksource: Switched to clocksource riscv_clocksource
> [    0.178581] workingset: timestamp_bits=62 max_order=11 bucket_order=0
> [    0.185846] 38000000.serial: ttySIF0 at MMIO 0x38000000 (irq = 1, base_baud = 0) is a SiFive UART v0
> [    0.194344] printk: console [ttySIF0] enabled
> [    0.194344] printk: console [ttySIF0] enabled
> [    0.202929] printk: bootconsole [sifive0] disabled
> [    0.202929] printk: bootconsole [sifive0] disabled
> [    0.214853] random: get_random_bytes called from 0x0000000080055178 with crng_init=0
> [    0.223606] devtmpfs: mounted
> [    0.226861] Freeing unused kernel memory: 320K
> [    0.230574] This architecture does not have kernel memory protection.
> [    0.236987] Run /sbin/init as init process
> [    0.241181] Run /etc/init as init process
> [    0.245178] Run /bin/init as init process
>
> -----------------------------
> | Kendryte K210 NOMMU Linux |
> -----------------------------
> Mounting /proc
> Starting shell
>
>
> BusyBox v1.32.0.git (2020-02-12 17:51:45 JST) hush - the humble shell
> Enter 'help' for a list of built-in commands.
>
> / # cat /proc/cpuinfo
> processor	: 0
> hart		: 0
> isa		: rv64imafdc
>
> processor	: 1
> hart		: 1
> isa		: rv64imafdc
>
> / #
> / # ls -l /
> drwxrwxr-x    2 1000     1000             0 Feb 12  2020 bin
> drwxr-xr-x    2 0        0                0 Jan  1 00:00 dev
> drwxrwxr-x    2 1000     1000             0 Feb 12  2020 etc
> dr-xr-xr-x   58 0        0                0 Jan  1 00:00 proc
> drwxrwxr-x    2 1000     1000             0 Feb 12  2020 root
> drwxrwxr-x    2 1000     1000             0 Feb 12  2020 sbin
> drwxrwxr-x    2 1000     1000             0 Feb 12  2020 sys
> drwxrwxr-x    2 1000     1000             0 Feb 12  2020 tmp
> drwxrwxr-x    4 1000     1000             0 Feb 12  2020 usr
> / #
> / # cat /proc/vmstat
> nr_free_pages 1148
> ...
> / #
>
> The K210 SoC has more devices (GPIO, SD-card, LCD, etc) that likely can
> be enabled and used. For this, the sysctl driver will need further
> improvements as each device uses a different clock. To share the fun,
> supporting these is left as an exercise for the hobbyist and hackers
> interested in this SoC/boards :)
>
> Christoph Hellwig (2):
>   riscv: Add Kendryte K210 SoC support
>   riscv: create a loader.bin for the kendryte kflash.py tool
>
> Damien Le Moal (8):
>   riscv: Fix gitignore
>   riscv: Force flat memory model with no-mmu
>   riscv: Unaligned load/store handling for M_MODE
>   riscv: Add BUILTIN_DTB support
>   riscv: Add SOC early init support
>   riscv: Select required drivers for Kendryte SOC
>   riscv: Add Kendryte K210 device tree
>   riscv: Kendryte K210 default config
>
>  arch/riscv/Kbuild                       |   1 +
>  arch/riscv/Kconfig                      |  19 ++
>  arch/riscv/Kconfig.socs                 |  10 +
>  arch/riscv/Makefile                     |   4 +-
>  arch/riscv/boot/.gitignore              |   2 +
>  arch/riscv/boot/Makefile                |   3 +
>  arch/riscv/boot/dts/Makefile            |   5 +
>  arch/riscv/boot/dts/kendryte/Makefile   |   2 +
>  arch/riscv/boot/dts/kendryte/k210.dts   |  23 ++
>  arch/riscv/boot/dts/kendryte/k210.dtsi  | 123 ++++++++
>  arch/riscv/configs/nommu_k210_defconfig |  68 +++++
>  arch/riscv/include/asm/soc.h            |  23 ++
>  arch/riscv/kernel/Makefile              |   3 +-
>  arch/riscv/kernel/head.S                |   1 +
>  arch/riscv/kernel/setup.c               |   6 +
>  arch/riscv/kernel/soc.c                 |  28 ++
>  arch/riscv/kernel/traps.c               |  27 +-
>  arch/riscv/kernel/traps_misaligned.c    | 371 ++++++++++++++++++++++++
>  arch/riscv/kernel/vmlinux.lds.S         |   6 +
>  arch/riscv/mm/init.c                    |   4 +
>  drivers/soc/Kconfig                     |   1 +
>  drivers/soc/Makefile                    |   1 +
>  drivers/soc/kendryte/Kconfig            |  14 +
>  drivers/soc/kendryte/Makefile           |   3 +
>  drivers/soc/kendryte/k210-sysctl.c      | 245 ++++++++++++++++
>  25 files changed, 987 insertions(+), 6 deletions(-)
>  create mode 100644 arch/riscv/boot/dts/kendryte/Makefile
>  create mode 100644 arch/riscv/boot/dts/kendryte/k210.dts
>  create mode 100644 arch/riscv/boot/dts/kendryte/k210.dtsi
>  create mode 100644 arch/riscv/configs/nommu_k210_defconfig
>  create mode 100644 arch/riscv/include/asm/soc.h
>  create mode 100644 arch/riscv/kernel/soc.c
>  create mode 100644 arch/riscv/kernel/traps_misaligned.c
>  create mode 100644 drivers/soc/kendryte/Kconfig
>  create mode 100644 drivers/soc/kendryte/Makefile
>  create mode 100644 drivers/soc/kendryte/k210-sysctl.c

[00/10] Kendryte k210 SoC boards support

Message

Comments