diff mbox series

ARM: dts: ux500: Reserve memory carveouts

Message ID 20201213225517.3838501-1-linus.walleij@linaro.org (mailing list archive)
State Queued
Headers show
Series ARM: dts: ux500: Reserve memory carveouts | expand

Commit Message

Linus Walleij Dec. 13, 2020, 10:55 p.m. UTC
The Ux500 platforms have some memory carveouts set aside for
communicating with the modem and for the initial secure software
(ISSW). These areas are protected by the memory controller
and will result in an external abort if accessed like common
read/write memory.

On the legacy boot loaders, these were set aside by using
cmdline arguments such as this:

  mem=96M@0 mem_mtrace=15M@96M mem_mshared=1M@111M
  mem_modem=16M@112M mali.mali_mem=32M@128M mem=96M@160M
  hwmem=127M@256M mem_issw=1M@383M mem_ram_console=1M@384M
  mem=638M@385M

Reserve the relevant areas in the device tree instead. The
"mali", "hwmem", "mem_ram_console" and the trailing 1MB at the
end of the memory reservations in the list are not relevant for
the upstream kernel as these are nowadays replaced with
upstream technologies such as CMA. The modem and ISSW
reservations are necessary.

This was manifested in a bug that surfaced in response to
commit 7fef431be9c9 ("mm/page_alloc: place pages to tail in __free_pages_core()")
which changes the behaviour of memory allocations
in such a way that the platform will sooner run into these
dangerous areas, with "Unhandled fault: imprecise external
abort (0xc06) at 0xb6fd83dc" or similar: the real reason
turns out to be that the PTE is pointing right into one of
the reserved memory areas. We were just lucky until now.

We need to augment the DB8500 and DB8520 SoCs similarly
and also create a new include for the DB9500 used in the
Snowball since this does not have a modem and thus does
not need the modem memory reservation, albeit it needs
the ISSW reservation.

Cc: stable@vger.kernel.org
Cc: David Hildenbrand <david@redhat.com>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
ARM SoC folks: please apply this directly for fixes.

David: just FYI if you run into more of these type of
regressions. Actually the patch is unintentionally good
at smoking out other bugs :D
---
 arch/arm/boot/dts/ste-db8500.dtsi  | 38 ++++++++++++++++++++++++++++++
 arch/arm/boot/dts/ste-db8520.dtsi  | 38 ++++++++++++++++++++++++++++++
 arch/arm/boot/dts/ste-db9500.dtsi  | 35 +++++++++++++++++++++++++++
 arch/arm/boot/dts/ste-snowball.dts |  2 +-
 4 files changed, 112 insertions(+), 1 deletion(-)
 create mode 100644 arch/arm/boot/dts/ste-db9500.dtsi

Comments

David Hildenbrand Dec. 14, 2020, 9:21 a.m. UTC | #1
On 13.12.20 23:55, Linus Walleij wrote:
> The Ux500 platforms have some memory carveouts set aside for
> communicating with the modem and for the initial secure software
> (ISSW). These areas are protected by the memory controller
> and will result in an external abort if accessed like common
> read/write memory.
> 
> On the legacy boot loaders, these were set aside by using
> cmdline arguments such as this:
> 
>   mem=96M@0 mem_mtrace=15M@96M mem_mshared=1M@111M
>   mem_modem=16M@112M mali.mali_mem=32M@128M mem=96M@160M
>   hwmem=127M@256M mem_issw=1M@383M mem_ram_console=1M@384M
>   mem=638M@385M
> 
> Reserve the relevant areas in the device tree instead. The
> "mali", "hwmem", "mem_ram_console" and the trailing 1MB at the
> end of the memory reservations in the list are not relevant for
> the upstream kernel as these are nowadays replaced with
> upstream technologies such as CMA. The modem and ISSW
> reservations are necessary.
> 
> This was manifested in a bug that surfaced in response to
> commit 7fef431be9c9 ("mm/page_alloc: place pages to tail in __free_pages_core()")
> which changes the behaviour of memory allocations
> in such a way that the platform will sooner run into these
> dangerous areas, with "Unhandled fault: imprecise external
> abort (0xc06) at 0xb6fd83dc" or similar: the real reason
> turns out to be that the PTE is pointing right into one of
> the reserved memory areas. We were just lucky until now.
> 
> We need to augment the DB8500 and DB8520 SoCs similarly
> and also create a new include for the DB9500 used in the
> Snowball since this does not have a modem and thus does
> not need the modem memory reservation, albeit it needs
> the ISSW reservation.
> 
> Cc: stable@vger.kernel.org
> Cc: David Hildenbrand <david@redhat.com>
> Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
> ---
> ARM SoC folks: please apply this directly for fixes.

Can we come up with a Fixes: tag or has this been broken forever?
(assuming modern boot loaders)

> 
> David: just FYI if you run into more of these type of
> regressions. Actually the patch is unintentionally good
> at smoking out other bugs :D

Thanks for CCing - I'm adding some people that ran into similar issues,
but not sure if the other bugreports are related (or have similar root
causes).

Thanks a lot!

> ---
>  arch/arm/boot/dts/ste-db8500.dtsi  | 38 ++++++++++++++++++++++++++++++
>  arch/arm/boot/dts/ste-db8520.dtsi  | 38 ++++++++++++++++++++++++++++++
>  arch/arm/boot/dts/ste-db9500.dtsi  | 35 +++++++++++++++++++++++++++
>  arch/arm/boot/dts/ste-snowball.dts |  2 +-
>  4 files changed, 112 insertions(+), 1 deletion(-)
>  create mode 100644 arch/arm/boot/dts/ste-db9500.dtsi
> 
> diff --git a/arch/arm/boot/dts/ste-db8500.dtsi b/arch/arm/boot/dts/ste-db8500.dtsi
> index d309fad32229..344d29853bf7 100644
> --- a/arch/arm/boot/dts/ste-db8500.dtsi
> +++ b/arch/arm/boot/dts/ste-db8500.dtsi
> @@ -12,4 +12,42 @@ cpu@300 {
>  					    200000 0>;
>  		};
>  	};
> +
> +	reserved-memory {
> +		#address-cells = <1>;
> +		#size-cells = <1>;
> +		ranges;
> +
> +		/* Modem trace memory */
> +		ram@06000000 {
> +			reg = <0x06000000 0x00f00000>;
> +			no-map;
> +		};
> +
> +		/* Modem shared memory */
> +		ram@06f00000 {
> +			reg = <0x06f00000 0x00100000>;
> +			no-map;
> +		};
> +
> +		/* Modem private memory */
> +		ram@07000000 {
> +			reg = <0x07000000 0x01000000>;
> +			no-map;
> +		};
> +
> +		/*
> +		 * Initial Secure Software ISSW memory
> +		 *
> +		 * This is probably only used if the kernel tries
> +		 * to actually call into trustzone to run secure
> +		 * applications, which the mainline kernel probably
> +		 * will not do on this old chipset. But you can never
> +		 * be too careful, so reserve this memory anyway.
> +		 */
> +		ram@17f00000 {
> +			reg = <0x17f00000 0x00100000>;
> +			no-map;
> +		};
> +	};
>  };
> diff --git a/arch/arm/boot/dts/ste-db8520.dtsi b/arch/arm/boot/dts/ste-db8520.dtsi
> index 48bd8728ae27..287804e9e183 100644
> --- a/arch/arm/boot/dts/ste-db8520.dtsi
> +++ b/arch/arm/boot/dts/ste-db8520.dtsi
> @@ -12,4 +12,42 @@ cpu@300 {
>  					    200000 0>;
>  		};
>  	};
> +
> +	reserved-memory {
> +		#address-cells = <1>;
> +		#size-cells = <1>;
> +		ranges;
> +
> +		/* Modem trace memory */
> +		ram@06000000 {
> +			reg = <0x06000000 0x00f00000>;
> +			no-map;
> +		};
> +
> +		/* Modem shared memory */
> +		ram@06f00000 {
> +			reg = <0x06f00000 0x00100000>;
> +			no-map;
> +		};
> +
> +		/* Modem private memory */
> +		ram@07000000 {
> +			reg = <0x07000000 0x01000000>;
> +			no-map;
> +		};
> +
> +		/*
> +		 * Initial Secure Software ISSW memory
> +		 *
> +		 * This is probably only used if the kernel tries
> +		 * to actually call into trustzone to run secure
> +		 * applications, which the mainline kernel probably
> +		 * will not do on this old chipset. But you can never
> +		 * be too careful, so reserve this memory anyway.
> +		 */
> +		ram@17f00000 {
> +			reg = <0x17f00000 0x00100000>;
> +			no-map;
> +		};
> +	};
>  };
> diff --git a/arch/arm/boot/dts/ste-db9500.dtsi b/arch/arm/boot/dts/ste-db9500.dtsi
> new file mode 100644
> index 000000000000..0afff703191c
> --- /dev/null
> +++ b/arch/arm/boot/dts/ste-db9500.dtsi
> @@ -0,0 +1,35 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +
> +#include "ste-dbx5x0.dtsi"
> +
> +/ {
> +	cpus {
> +		cpu@300 {
> +			/* cpufreq controls */
> +			operating-points = <1152000 0
> +					    800000 0
> +					    400000 0
> +					    200000 0>;
> +		};
> +	};
> +
> +	reserved-memory {
> +		#address-cells = <1>;
> +		#size-cells = <1>;
> +		ranges;
> +
> +		/*
> +		 * Initial Secure Software ISSW memory
> +		 *
> +		 * This is probably only used if the kernel tries
> +		 * to actually call into trustzone to run secure
> +		 * applications, which the mainline kernel probably
> +		 * will not do on this old chipset. But you can never
> +		 * be too careful, so reserve this memory anyway.
> +		 */
> +		ram@17f00000 {
> +			reg = <0x17f00000 0x00100000>;
> +			no-map;
> +		};
> +	};
> +};
> diff --git a/arch/arm/boot/dts/ste-snowball.dts b/arch/arm/boot/dts/ste-snowball.dts
> index be90e73c923e..27d8a07718a0 100644
> --- a/arch/arm/boot/dts/ste-snowball.dts
> +++ b/arch/arm/boot/dts/ste-snowball.dts
> @@ -4,7 +4,7 @@
>   */
>  
>  /dts-v1/;
> -#include "ste-db8500.dtsi"
> +#include "ste-db9500.dtsi"
>  #include "ste-href-ab8500.dtsi"
>  #include "ste-href-family-pinctrl.dtsi"
>  
>
Linus Walleij Dec. 14, 2020, 9:27 a.m. UTC | #2
On Mon, Dec 14, 2020 at 10:21 AM David Hildenbrand <david@redhat.com> wrote:

> > ARM SoC folks: please apply this directly for fixes.
>
> Can we come up with a Fixes: tag or has this been broken forever?
> (assuming modern boot loaders)

It's been broken forever :/

> > David: just FYI if you run into more of these type of
> > regressions. Actually the patch is unintentionally good
> > at smoking out other bugs :D
>
> Thanks for CCing - I'm adding some people that ran into similar issues,
> but not sure if the other bugreports are related (or have similar root
> causes).

Yeah we first were convinced there was something wrong with
the patch you made but I read it over and over again and there
is nothing wrong with it at all. It just alters the behaviour pattern of
memory management in some apparently drastic ways.

After a lot of silent crashes I finally got an external abort with
a reasonable backtrace showing the PTE pointing to this
modem memory and then we figured it out.

Yours,
Linus Walleij
Kalle Valo Dec. 14, 2020, 10:02 a.m. UTC | #3
Linus Walleij <linus.walleij@linaro.org> writes:

> On Mon, Dec 14, 2020 at 10:21 AM David Hildenbrand <david@redhat.com> wrote:
>
>> > ARM SoC folks: please apply this directly for fixes.
>>
>> Can we come up with a Fixes: tag or has this been broken forever?
>> (assuming modern boot loaders)
>
> It's been broken forever :/
>
>> > David: just FYI if you run into more of these type of
>> > regressions. Actually the patch is unintentionally good
>> > at smoking out other bugs :D
>>
>> Thanks for CCing - I'm adding some people that ran into similar issues,
>> but not sure if the other bugreports are related (or have similar root
>> causes).
>
> Yeah we first were convinced there was something wrong with
> the patch you made but I read it over and over again and there
> is nothing wrong with it at all. It just alters the behaviour pattern of
> memory management in some apparently drastic ways.
>
> After a lot of silent crashes I finally got an external abort with
> a reasonable backtrace showing the PTE pointing to this
> modem memory and then we figured it out.

We had similar experiences with ath11k (Wi-Fi 6) and QCA6390 firmware.
So indeed commit 7fef431be9c9 ("mm/page_alloc: place pages to tail in
__free_pages_core()") is a great way to catch odd firmware or hardware
problems, which most likely would have gone unnoticed otherwise and
users would have end up experiencing random crashes.
Arnd Bergmann Jan. 15, 2021, 5:22 p.m. UTC | #4
From: Arnd Bergmann <arnd@arndb.de>

On Sun, 13 Dec 2020 23:55:17 +0100, Linus Walleij wrote:
> The Ux500 platforms have some memory carveouts set aside for
> communicating with the modem and for the initial secure software
> (ISSW). These areas are protected by the memory controller
> and will result in an external abort if accessed like common
> read/write memory.
> 
> On the legacy boot loaders, these were set aside by using
> cmdline arguments such as this:
> 
> [...]

I had mistakenly not applied this last year before the merge
window, as I misread the description as being not very urgent.

On second look, this seems important enough for the v5.11
fixes branch, so I applied it now.

[1/1] ARM: dts: ux500: Reserve memory carveouts
      commit: 8a996b2d8a03beae3cb6adfc12673778c192085d

       Arnd
Linus Walleij Jan. 15, 2021, 10:21 p.m. UTC | #5
On Fri, Jan 15, 2021 at 6:22 PM Arnd Bergmann <arnd@kernel.org> wrote:

> I had mistakenly not applied this last year before the merge
> window, as I misread the description as being not very urgent.
>
> On second look, this seems important enough for the v5.11
> fixes branch, so I applied it now.

I was just thinking about it while cooking dinner!
- Hey what with that critical patch now again...

Thanks a lot!
Linus
diff mbox series

Patch

diff --git a/arch/arm/boot/dts/ste-db8500.dtsi b/arch/arm/boot/dts/ste-db8500.dtsi
index d309fad32229..344d29853bf7 100644
--- a/arch/arm/boot/dts/ste-db8500.dtsi
+++ b/arch/arm/boot/dts/ste-db8500.dtsi
@@ -12,4 +12,42 @@  cpu@300 {
 					    200000 0>;
 		};
 	};
+
+	reserved-memory {
+		#address-cells = <1>;
+		#size-cells = <1>;
+		ranges;
+
+		/* Modem trace memory */
+		ram@06000000 {
+			reg = <0x06000000 0x00f00000>;
+			no-map;
+		};
+
+		/* Modem shared memory */
+		ram@06f00000 {
+			reg = <0x06f00000 0x00100000>;
+			no-map;
+		};
+
+		/* Modem private memory */
+		ram@07000000 {
+			reg = <0x07000000 0x01000000>;
+			no-map;
+		};
+
+		/*
+		 * Initial Secure Software ISSW memory
+		 *
+		 * This is probably only used if the kernel tries
+		 * to actually call into trustzone to run secure
+		 * applications, which the mainline kernel probably
+		 * will not do on this old chipset. But you can never
+		 * be too careful, so reserve this memory anyway.
+		 */
+		ram@17f00000 {
+			reg = <0x17f00000 0x00100000>;
+			no-map;
+		};
+	};
 };
diff --git a/arch/arm/boot/dts/ste-db8520.dtsi b/arch/arm/boot/dts/ste-db8520.dtsi
index 48bd8728ae27..287804e9e183 100644
--- a/arch/arm/boot/dts/ste-db8520.dtsi
+++ b/arch/arm/boot/dts/ste-db8520.dtsi
@@ -12,4 +12,42 @@  cpu@300 {
 					    200000 0>;
 		};
 	};
+
+	reserved-memory {
+		#address-cells = <1>;
+		#size-cells = <1>;
+		ranges;
+
+		/* Modem trace memory */
+		ram@06000000 {
+			reg = <0x06000000 0x00f00000>;
+			no-map;
+		};
+
+		/* Modem shared memory */
+		ram@06f00000 {
+			reg = <0x06f00000 0x00100000>;
+			no-map;
+		};
+
+		/* Modem private memory */
+		ram@07000000 {
+			reg = <0x07000000 0x01000000>;
+			no-map;
+		};
+
+		/*
+		 * Initial Secure Software ISSW memory
+		 *
+		 * This is probably only used if the kernel tries
+		 * to actually call into trustzone to run secure
+		 * applications, which the mainline kernel probably
+		 * will not do on this old chipset. But you can never
+		 * be too careful, so reserve this memory anyway.
+		 */
+		ram@17f00000 {
+			reg = <0x17f00000 0x00100000>;
+			no-map;
+		};
+	};
 };
diff --git a/arch/arm/boot/dts/ste-db9500.dtsi b/arch/arm/boot/dts/ste-db9500.dtsi
new file mode 100644
index 000000000000..0afff703191c
--- /dev/null
+++ b/arch/arm/boot/dts/ste-db9500.dtsi
@@ -0,0 +1,35 @@ 
+// SPDX-License-Identifier: GPL-2.0-or-later
+
+#include "ste-dbx5x0.dtsi"
+
+/ {
+	cpus {
+		cpu@300 {
+			/* cpufreq controls */
+			operating-points = <1152000 0
+					    800000 0
+					    400000 0
+					    200000 0>;
+		};
+	};
+
+	reserved-memory {
+		#address-cells = <1>;
+		#size-cells = <1>;
+		ranges;
+
+		/*
+		 * Initial Secure Software ISSW memory
+		 *
+		 * This is probably only used if the kernel tries
+		 * to actually call into trustzone to run secure
+		 * applications, which the mainline kernel probably
+		 * will not do on this old chipset. But you can never
+		 * be too careful, so reserve this memory anyway.
+		 */
+		ram@17f00000 {
+			reg = <0x17f00000 0x00100000>;
+			no-map;
+		};
+	};
+};
diff --git a/arch/arm/boot/dts/ste-snowball.dts b/arch/arm/boot/dts/ste-snowball.dts
index be90e73c923e..27d8a07718a0 100644
--- a/arch/arm/boot/dts/ste-snowball.dts
+++ b/arch/arm/boot/dts/ste-snowball.dts
@@ -4,7 +4,7 @@ 
  */
 
 /dts-v1/;
-#include "ste-db8500.dtsi"
+#include "ste-db9500.dtsi"
 #include "ste-href-ab8500.dtsi"
 #include "ste-href-family-pinctrl.dtsi"