[2/2] ARM: Allow either FLATMEM or SPARSEMEM on the multiplatform build

Commit Message

Florian Fainelli May 6, 2020, 11:50 p.m. UTC

From: Gregory Fong <[e-mail]>

ARMv7 chips with LPAE can often benefit from SPARSEMEM, as portions of
system memory can be located deep in the 36-bit address space.  Allow
FLATMEM or SPARSEMEM to be selectable at compile time; FLATMEM remains
the default.

This is based on Kevin's "[PATCH 3/3] ARM: Allow either FLATMEM or
SPARSEMEM on the multi-v7 build" from [1] and shamelessly rips off his
commit message text above.  As Arnd pointed out at [2] there doesn't
seem to be any reason to tie this specifically to ARMv7, so this has
been changed to apply to all multiplatform kernels.

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-September/286837.html
[2] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-October/298950.html

Cc: Kevin Cernekee <[e-mail]>
Tested-by: Stephen Boyd <[e-mail]>
Signed-off-by: Gregory Fong <[e-mail]>
Signed-off-by: Doug Berger <[e-mail]>
Signed-off-by: Florian Fainelli <[e-mail]>
---
 arch/arm/Kconfig | 5 +++++
 1 file changed, 5 insertions(+)

Comments

Mike Rapoport May 7, 2020, 7:27 a.m. UTC | #1

On Wed, May 06, 2020 at 04:50:09PM -0700, Florian Fainelli wrote:
> From: Gregory Fong <[e-mail]>
> 
> ARMv7 chips with LPAE can often benefit from SPARSEMEM, as portions of
> system memory can be located deep in the 36-bit address space.  Allow
> FLATMEM or SPARSEMEM to be selectable at compile time; FLATMEM remains
> the default.
> 
> This is based on Kevin's "[PATCH 3/3] ARM: Allow either FLATMEM or
> SPARSEMEM on the multi-v7 build" from [1] and shamelessly rips off his
> commit message text above.  As Arnd pointed out at [2] there doesn't
> seem to be any reason to tie this specifically to ARMv7, so this has
> been changed to apply to all multiplatform kernels.
> 
> [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-September/286837.html
> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-October/298950.html
> 
> Cc: Kevin Cernekee <[e-mail]>
> Tested-by: Stephen Boyd <[e-mail]>
> Signed-off-by: Gregory Fong <[e-mail]>
> Signed-off-by: Doug Berger <[e-mail]>
> Signed-off-by: Florian Fainelli <[e-mail]>
> ---
>  arch/arm/Kconfig | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index 5986277296c3..7bb5264a9c3a 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -312,6 +312,8 @@ choice
>  config ARCH_MULTIPLATFORM
>  	bool "Allow multiple platforms to be selected"
>  	depends on MMU
> +	select ARCH_FLATMEM_ENABLE
> +	select ARCH_SPARSEMEM_ENABLE

The logic in mm/Kconfig is quite convoluted, so selecting
ARCH_SPARSEMEM_ENABLE will automatically make SPARSEMEM the only option.

On top of this you would need to enable ARCH_SELECT_MEMORY_MODEL, e.g.
something like:

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 7bb5264a9c3a..b6eb1a28ca27 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -314,6 +314,7 @@ config ARCH_MULTIPLATFORM
 	depends on MMU
 	select ARCH_FLATMEM_ENABLE
 	select ARCH_SPARSEMEM_ENABLE
+	select ARCH_SELECT_MEMORY_MODEL
 	select ARM_HAS_SG_CHAIN
 	select ARM_PATCH_PHYS_VIRT
 	select AUTO_ZRELADDR
@@ -1523,6 +1524,9 @@ config ARCH_FLATMEM_ENABLE
 config ARCH_SPARSEMEM_ENABLE
 	bool
 
+config ARCH_SELECT_MEMORY_MODEL
+	bool
+
 config HAVE_ARCH_PFN_VALID
 	def_bool ARCH_HAS_HOLES_MEMORYMODEL || !SPARSEMEM
 
>  	select ARM_HAS_SG_CHAIN
>  	select ARM_PATCH_PHYS_VIRT
>  	select AUTO_ZRELADDR
> @@ -1515,6 +1517,9 @@ config OABI_COMPAT
>  config ARCH_HAS_HOLES_MEMORYMODEL
>  	bool
>  
> +config ARCH_FLATMEM_ENABLE
> +	bool
> +
>  config ARCH_SPARSEMEM_ENABLE
>  	bool
>  
> -- 
> 2.17.1
>

Florian Fainelli May 7, 2020, 8:11 p.m. UTC | #2

On 5/7/2020 12:27 AM, Mike Rapoport wrote:
> On Wed, May 06, 2020 at 04:50:09PM -0700, Florian Fainelli wrote:
>> From: Gregory Fong <[e-mail]>
>>
>> ARMv7 chips with LPAE can often benefit from SPARSEMEM, as portions of
>> system memory can be located deep in the 36-bit address space.  Allow
>> FLATMEM or SPARSEMEM to be selectable at compile time; FLATMEM remains
>> the default.
>>
>> This is based on Kevin's "[PATCH 3/3] ARM: Allow either FLATMEM or
>> SPARSEMEM on the multi-v7 build" from [1] and shamelessly rips off his
>> commit message text above.  As Arnd pointed out at [2] there doesn't
>> seem to be any reason to tie this specifically to ARMv7, so this has
>> been changed to apply to all multiplatform kernels.
>>
>> [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-September/286837.html
>> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-October/298950.html
>>
>> Cc: Kevin Cernekee <[e-mail]>
>> Tested-by: Stephen Boyd <[e-mail]>
>> Signed-off-by: Gregory Fong <[e-mail]>
>> Signed-off-by: Doug Berger <[e-mail]>
>> Signed-off-by: Florian Fainelli <[e-mail]>
>> ---
>>  arch/arm/Kconfig | 5 +++++
>>  1 file changed, 5 insertions(+)
>>
>> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
>> index 5986277296c3..7bb5264a9c3a 100644
>> --- a/arch/arm/Kconfig
>> +++ b/arch/arm/Kconfig
>> @@ -312,6 +312,8 @@ choice
>>  config ARCH_MULTIPLATFORM
>>  	bool "Allow multiple platforms to be selected"
>>  	depends on MMU
>> +	select ARCH_FLATMEM_ENABLE
>> +	select ARCH_SPARSEMEM_ENABLE
> 
> The logic in mm/Kconfig is quite convoluted, so selecting
> ARCH_SPARSEMEM_ENABLE will automatically make SPARSEMEM the only option.
> 
> On top of this you would need to enable ARCH_SELECT_MEMORY_MODEL, e.g.
> something like:

Yes indeed, thanks that does allow me to select between flatmem and
sparsemem from menuconfig correctly now.

Florian Fainelli May 18, 2020, 3:58 p.m. UTC | #3

On 5/7/2020 1:11 PM, Florian Fainelli wrote:
> 
> 
> On 5/7/2020 12:27 AM, Mike Rapoport wrote:
>> On Wed, May 06, 2020 at 04:50:09PM -0700, Florian Fainelli wrote:
>>> From: Gregory Fong <[e-mail]>
>>>
>>> ARMv7 chips with LPAE can often benefit from SPARSEMEM, as portions of
>>> system memory can be located deep in the 36-bit address space.  Allow
>>> FLATMEM or SPARSEMEM to be selectable at compile time; FLATMEM remains
>>> the default.
>>>
>>> This is based on Kevin's "[PATCH 3/3] ARM: Allow either FLATMEM or
>>> SPARSEMEM on the multi-v7 build" from [1] and shamelessly rips off his
>>> commit message text above.  As Arnd pointed out at [2] there doesn't
>>> seem to be any reason to tie this specifically to ARMv7, so this has
>>> been changed to apply to all multiplatform kernels.
>>>
>>> [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-September/286837.html
>>> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-October/298950.html
>>>
>>> Cc: Kevin Cernekee <[e-mail]>
>>> Tested-by: Stephen Boyd <[e-mail]>
>>> Signed-off-by: Gregory Fong <[e-mail]>
>>> Signed-off-by: Doug Berger <[e-mail]>
>>> Signed-off-by: Florian Fainelli <[e-mail]>
>>> ---
>>>  arch/arm/Kconfig | 5 +++++
>>>  1 file changed, 5 insertions(+)
>>>
>>> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
>>> index 5986277296c3..7bb5264a9c3a 100644
>>> --- a/arch/arm/Kconfig
>>> +++ b/arch/arm/Kconfig
>>> @@ -312,6 +312,8 @@ choice
>>>  config ARCH_MULTIPLATFORM
>>>  	bool "Allow multiple platforms to be selected"
>>>  	depends on MMU
>>> +	select ARCH_FLATMEM_ENABLE
>>> +	select ARCH_SPARSEMEM_ENABLE
>>
>> The logic in mm/Kconfig is quite convoluted, so selecting
>> ARCH_SPARSEMEM_ENABLE will automatically make SPARSEMEM the only option.
>>
>> On top of this you would need to enable ARCH_SELECT_MEMORY_MODEL, e.g.
>> something like:
> 
> Yes indeed, thanks that does allow me to select between flatmem and
> sparsemem from menuconfig correctly now.

Mike, do you want to make a formal submission to Russell's patch
tracker? If so, feel free to add:

Tested-by: Florian Fainelli <[e-mail]>

Thanks!
--
Florian

Mike Rapoport May 18, 2020, 7:45 p.m. UTC | #4

On Mon, May 18, 2020 at 08:58:36AM -0700, Florian Fainelli wrote:
> 
> 
> On 5/7/2020 1:11 PM, Florian Fainelli wrote:
> > 
> > 
> > On 5/7/2020 12:27 AM, Mike Rapoport wrote:
> >> On Wed, May 06, 2020 at 04:50:09PM -0700, Florian Fainelli wrote:
> >>> From: Gregory Fong <[e-mail]>
> >>>
> >>> ARMv7 chips with LPAE can often benefit from SPARSEMEM, as portions of
> >>> system memory can be located deep in the 36-bit address space.  Allow
> >>> FLATMEM or SPARSEMEM to be selectable at compile time; FLATMEM remains
> >>> the default.
> >>>
> >>> This is based on Kevin's "[PATCH 3/3] ARM: Allow either FLATMEM or
> >>> SPARSEMEM on the multi-v7 build" from [1] and shamelessly rips off his
> >>> commit message text above.  As Arnd pointed out at [2] there doesn't
> >>> seem to be any reason to tie this specifically to ARMv7, so this has
> >>> been changed to apply to all multiplatform kernels.
> >>>
> >>> [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-September/286837.html
> >>> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-October/298950.html
> >>>
> >>> Cc: Kevin Cernekee <[e-mail]>
> >>> Tested-by: Stephen Boyd <[e-mail]>
> >>> Signed-off-by: Gregory Fong <[e-mail]>
> >>> Signed-off-by: Doug Berger <[e-mail]>
> >>> Signed-off-by: Florian Fainelli <[e-mail]>
> >>> ---
> >>>  arch/arm/Kconfig | 5 +++++
> >>>  1 file changed, 5 insertions(+)
> >>>
> >>> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> >>> index 5986277296c3..7bb5264a9c3a 100644
> >>> --- a/arch/arm/Kconfig
> >>> +++ b/arch/arm/Kconfig
> >>> @@ -312,6 +312,8 @@ choice
> >>>  config ARCH_MULTIPLATFORM
> >>>  	bool "Allow multiple platforms to be selected"
> >>>  	depends on MMU
> >>> +	select ARCH_FLATMEM_ENABLE
> >>> +	select ARCH_SPARSEMEM_ENABLE
> >>
> >> The logic in mm/Kconfig is quite convoluted, so selecting
> >> ARCH_SPARSEMEM_ENABLE will automatically make SPARSEMEM the only option.
> >>
> >> On top of this you would need to enable ARCH_SELECT_MEMORY_MODEL, e.g.
> >> something like:
> > 
> > Yes indeed, thanks that does allow me to select between flatmem and
> > sparsemem from menuconfig correctly now.
> 
> Mike, do you want to make a formal submission to Russell's patch
> tracker? If so, feel free to add:

I actually hoped to hear from people what do they think about switching
over to SPARSEMEM for the multiplatform builds. I think at least v7 CPUs
it would make sense.

Russel, Arnd, can you comment please?

> Tested-by: Florian Fainelli <[e-mail]>
> 
> Thanks!
> --
> Florian

Arnd Bergmann May 19, 2020, 7:59 a.m. UTC | #5

On Mon, May 18, 2020 at 9:45 PM Mike Rapoport <[e-mail]> wrote:
> On Mon, May 18, 2020 at 08:58:36AM -0700, Florian Fainelli wrote:
> > On 5/7/2020 1:11 PM, Florian Fainelli wrote:
> > > On 5/7/2020 12:27 AM, Mike Rapoport wrote:
> > >> On Wed, May 06, 2020 at 04:50:09PM -0700, Florian Fainelli wrote:
> > >>> From: Gregory Fong <[e-mail]>
> > >>>
> > >>> ARMv7 chips with LPAE can often benefit from SPARSEMEM, as portions of
> > >>> system memory can be located deep in the 36-bit address space.  Allow
> > >>> FLATMEM or SPARSEMEM to be selectable at compile time; FLATMEM remains
> > >>> the default.
> > >>>
> > >>> This is based on Kevin's "[PATCH 3/3] ARM: Allow either FLATMEM or
> > >>> SPARSEMEM on the multi-v7 build" from [1] and shamelessly rips off his
> > >>> commit message text above.  As Arnd pointed out at [2] there doesn't
> > >>> seem to be any reason to tie this specifically to ARMv7, so this has
> > >>> been changed to apply to all multiplatform kernels.
> > >>>
> > >>> [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-September/286837.html
> > >>> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-October/298950.html
> > >>>
> > >>> Cc: Kevin Cernekee <[e-mail]>
> > >>> Tested-by: Stephen Boyd <[e-mail]>
> > >>> Signed-off-by: Gregory Fong <[e-mail]>
> > >>> Signed-off-by: Doug Berger <[e-mail]>
> > >>> Signed-off-by: Florian Fainelli <[e-mail]>
> > >>> ---
> > >>>  arch/arm/Kconfig | 5 +++++
> > >>>  1 file changed, 5 insertions(+)
> > >>>
> > >>> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> > >>> index 5986277296c3..7bb5264a9c3a 100644
> > >>> --- a/arch/arm/Kconfig
> > >>> +++ b/arch/arm/Kconfig
> > >>> @@ -312,6 +312,8 @@ choice
> > >>>  config ARCH_MULTIPLATFORM
> > >>>   bool "Allow multiple platforms to be selected"
> > >>>   depends on MMU
> > >>> + select ARCH_FLATMEM_ENABLE
> > >>> + select ARCH_SPARSEMEM_ENABLE
> > >>
> > >> The logic in mm/Kconfig is quite convoluted, so selecting
> > >> ARCH_SPARSEMEM_ENABLE will automatically make SPARSEMEM the only option.
> > >>
> > >> On top of this you would need to enable ARCH_SELECT_MEMORY_MODEL, e.g.
> > >> something like:
> > >
> > > Yes indeed, thanks that does allow me to select between flatmem and
> > > sparsemem from menuconfig correctly now.
> >
> > Mike, do you want to make a formal submission to Russell's patch
> > tracker? If so, feel free to add:
>
> I actually hoped to hear from people what do they think about switching
> over to SPARSEMEM for the multiplatform builds. I think at least v7 CPUs
> it would make sense.

It seems that at least s5p/exynos and clps711x (armv4) used to default to
sparsemem, and that got lost in the multiplatform conversion.

I also see discontiguous memory ranges in multiple broadcom chips,
on TI dm8168, ecx-2000 and imx.

> Russel, Arnd, can you comment please?

I see no problem with giving users the choice for all multiplatform
builds. No idea on what the default should be really, i.e. whether
only v7 configurations should make it the default, or rather none of
them or all of them.

Maybe lets leave the default unchanged with flatmem but enable it
in multi_v7_defconfig and the configurations for chips that are known
to have discontiguous memory (clps, bcm, imx, exynos, ...).

      Arnd

Mike Rapoport May 19, 2020, 2:43 p.m. UTC | #6

On Tue, May 19, 2020 at 09:59:48AM +0200, Arnd Bergmann wrote:
> On Mon, May 18, 2020 at 9:45 PM Mike Rapoport <[e-mail]> wrote:
> > On Mon, May 18, 2020 at 08:58:36AM -0700, Florian Fainelli wrote:
> > > On 5/7/2020 1:11 PM, Florian Fainelli wrote:
> > > > On 5/7/2020 12:27 AM, Mike Rapoport wrote:
> > > >> On Wed, May 06, 2020 at 04:50:09PM -0700, Florian Fainelli wrote:
> > > >>> From: Gregory Fong <[e-mail]>
> > > >>>
> > > >>> ARMv7 chips with LPAE can often benefit from SPARSEMEM, as portions of
> > > >>> system memory can be located deep in the 36-bit address space.  Allow
> > > >>> FLATMEM or SPARSEMEM to be selectable at compile time; FLATMEM remains
> > > >>> the default.
> > > >>>
> > > >>> This is based on Kevin's "[PATCH 3/3] ARM: Allow either FLATMEM or
> > > >>> SPARSEMEM on the multi-v7 build" from [1] and shamelessly rips off his
> > > >>> commit message text above.  As Arnd pointed out at [2] there doesn't
> > > >>> seem to be any reason to tie this specifically to ARMv7, so this has
> > > >>> been changed to apply to all multiplatform kernels.
> > > >>>
> > > >>> [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-September/286837.html
> > > >>> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-October/298950.html
> > > >>>
> > > >>> Cc: Kevin Cernekee <[e-mail]>
> > > >>> Tested-by: Stephen Boyd <[e-mail]>
> > > >>> Signed-off-by: Gregory Fong <[e-mail]>
> > > >>> Signed-off-by: Doug Berger <[e-mail]>
> > > >>> Signed-off-by: Florian Fainelli <[e-mail]>
> > > >>> ---
> > > >>>  arch/arm/Kconfig | 5 +++++
> > > >>>  1 file changed, 5 insertions(+)
> > > >>>
> > > >>> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> > > >>> index 5986277296c3..7bb5264a9c3a 100644
> > > >>> --- a/arch/arm/Kconfig
> > > >>> +++ b/arch/arm/Kconfig
> > > >>> @@ -312,6 +312,8 @@ choice
> > > >>>  config ARCH_MULTIPLATFORM
> > > >>>   bool "Allow multiple platforms to be selected"
> > > >>>   depends on MMU
> > > >>> + select ARCH_FLATMEM_ENABLE
> > > >>> + select ARCH_SPARSEMEM_ENABLE
> > > >>
> > > >> The logic in mm/Kconfig is quite convoluted, so selecting
> > > >> ARCH_SPARSEMEM_ENABLE will automatically make SPARSEMEM the only option.
> > > >>
> > > >> On top of this you would need to enable ARCH_SELECT_MEMORY_MODEL, e.g.
> > > >> something like:
> > > >
> > > > Yes indeed, thanks that does allow me to select between flatmem and
> > > > sparsemem from menuconfig correctly now.
> > >
> > > Mike, do you want to make a formal submission to Russell's patch
> > > tracker? If so, feel free to add:
> >
> > I actually hoped to hear from people what do they think about switching
> > over to SPARSEMEM for the multiplatform builds. I think at least v7 CPUs
> > it would make sense.
> 
> It seems that at least s5p/exynos and clps711x (armv4) used to default to
> sparsemem, and that got lost in the multiplatform conversion.
> 
> I also see discontiguous memory ranges in multiple broadcom chips,
> on TI dm8168, ecx-2000 and imx.
> 
> > Russel, Arnd, can you comment please?
> 
> I see no problem with giving users the choice for all multiplatform
> builds. No idea on what the default should be really, i.e. whether
> only v7 configurations should make it the default, or rather none of
> them or all of them.

Me neither :)

I'd say let's move forward with letting user select the memory model for
all multiplaform builds and keep flatmem the default.

Enabling sparsemem (with SPARSEMEM_STATIC) for defconfig build increases
the kernel nearly by 8k and most probably will make things slower.


> Maybe lets leave the default unchanged with flatmem but enable it in
> multi_v7_defconfig and the configurations for chips that are known
> to have discontiguous memory (clps, bcm, imx, exynos, ...).
>
>       Arnd

Russell King (Oracle) May 19, 2020, 3:04 p.m. UTC | #7

On Tue, May 19, 2020 at 05:43:06PM +0300, Mike Rapoport wrote:
> On Tue, May 19, 2020 at 09:59:48AM +0200, Arnd Bergmann wrote:
> > On Mon, May 18, 2020 at 9:45 PM Mike Rapoport <[e-mail]> wrote:
> > > On Mon, May 18, 2020 at 08:58:36AM -0700, Florian Fainelli wrote:
> > > > On 5/7/2020 1:11 PM, Florian Fainelli wrote:
> > > > > On 5/7/2020 12:27 AM, Mike Rapoport wrote:
> > > > >> On Wed, May 06, 2020 at 04:50:09PM -0700, Florian Fainelli wrote:
> > > > >>> From: Gregory Fong <[e-mail]>
> > > > >>>
> > > > >>> ARMv7 chips with LPAE can often benefit from SPARSEMEM, as portions of
> > > > >>> system memory can be located deep in the 36-bit address space.  Allow
> > > > >>> FLATMEM or SPARSEMEM to be selectable at compile time; FLATMEM remains
> > > > >>> the default.
> > > > >>>
> > > > >>> This is based on Kevin's "[PATCH 3/3] ARM: Allow either FLATMEM or
> > > > >>> SPARSEMEM on the multi-v7 build" from [1] and shamelessly rips off his
> > > > >>> commit message text above.  As Arnd pointed out at [2] there doesn't
> > > > >>> seem to be any reason to tie this specifically to ARMv7, so this has
> > > > >>> been changed to apply to all multiplatform kernels.
> > > > >>>
> > > > >>> [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-September/286837.html
> > > > >>> [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-October/298950.html
> > > > >>>
> > > > >>> Cc: Kevin Cernekee <[e-mail]>
> > > > >>> Tested-by: Stephen Boyd <[e-mail]>
> > > > >>> Signed-off-by: Gregory Fong <[e-mail]>
> > > > >>> Signed-off-by: Doug Berger <[e-mail]>
> > > > >>> Signed-off-by: Florian Fainelli <[e-mail]>
> > > > >>> ---
> > > > >>>  arch/arm/Kconfig | 5 +++++
> > > > >>>  1 file changed, 5 insertions(+)
> > > > >>>
> > > > >>> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> > > > >>> index 5986277296c3..7bb5264a9c3a 100644
> > > > >>> --- a/arch/arm/Kconfig
> > > > >>> +++ b/arch/arm/Kconfig
> > > > >>> @@ -312,6 +312,8 @@ choice
> > > > >>>  config ARCH_MULTIPLATFORM
> > > > >>>   bool "Allow multiple platforms to be selected"
> > > > >>>   depends on MMU
> > > > >>> + select ARCH_FLATMEM_ENABLE
> > > > >>> + select ARCH_SPARSEMEM_ENABLE
> > > > >>
> > > > >> The logic in mm/Kconfig is quite convoluted, so selecting
> > > > >> ARCH_SPARSEMEM_ENABLE will automatically make SPARSEMEM the only option.
> > > > >>
> > > > >> On top of this you would need to enable ARCH_SELECT_MEMORY_MODEL, e.g.
> > > > >> something like:
> > > > >
> > > > > Yes indeed, thanks that does allow me to select between flatmem and
> > > > > sparsemem from menuconfig correctly now.
> > > >
> > > > Mike, do you want to make a formal submission to Russell's patch
> > > > tracker? If so, feel free to add:
> > >
> > > I actually hoped to hear from people what do they think about switching
> > > over to SPARSEMEM for the multiplatform builds. I think at least v7 CPUs
> > > it would make sense.
> > 
> > It seems that at least s5p/exynos and clps711x (armv4) used to default to
> > sparsemem, and that got lost in the multiplatform conversion.
> > 
> > I also see discontiguous memory ranges in multiple broadcom chips,
> > on TI dm8168, ecx-2000 and imx.
> > 
> > > Russel, Arnd, can you comment please?
> > 
> > I see no problem with giving users the choice for all multiplatform
> > builds. No idea on what the default should be really, i.e. whether
> > only v7 configurations should make it the default, or rather none of
> > them or all of them.
> 
> Me neither :)
> 
> I'd say let's move forward with letting user select the memory model for
> all multiplaform builds and keep flatmem the default.
> 
> Enabling sparsemem (with SPARSEMEM_STATIC) for defconfig build increases
> the kernel nearly by 8k and most probably will make things slower.

I think when selecting the default, you need to consider whether
platforms can boot with that default configuration or not.

There are cases where the reason to use sparsemem is not an optional
one but is out of necessity - they require the page array to be split
up in order to boot successfully.

With that in mind, flatmem becomes an "optimisation" over sparsemem.

Mike Rapoport May 19, 2020, 3:27 p.m. UTC | #8

On Tue, May 19, 2020 at 04:04:52PM +0100, Russell King - ARM Linux admin wrote:
> On Tue, May 19, 2020 at 05:43:06PM +0300, Mike Rapoport wrote:
> > On Tue, May 19, 2020 at 09:59:48AM +0200, Arnd Bergmann wrote:
> > > On Mon, May 18, 2020 at 9:45 PM Mike Rapoport <[e-mail]> wrote:
> > > > On Mon, May 18, 2020 at 08:58:36AM -0700, Florian Fainelli wrote:
> > > > >
> > > > > Mike, do you want to make a formal submission to Russell's patch
> > > > > tracker? If so, feel free to add:
> > > >
> > > > I actually hoped to hear from people what do they think about switching
> > > > over to SPARSEMEM for the multiplatform builds. I think at least v7 CPUs
> > > > it would make sense.
> > > 
> > > It seems that at least s5p/exynos and clps711x (armv4) used to default to
> > > sparsemem, and that got lost in the multiplatform conversion.
> > > 
> > > I also see discontiguous memory ranges in multiple broadcom chips,
> > > on TI dm8168, ecx-2000 and imx.
> > > 
> > > > Russel, Arnd, can you comment please?
> > > 
> > > I see no problem with giving users the choice for all multiplatform
> > > builds. No idea on what the default should be really, i.e. whether
> > > only v7 configurations should make it the default, or rather none of
> > > them or all of them.
> > 
> > Me neither :)
> > 
> > I'd say let's move forward with letting user select the memory model for
> > all multiplaform builds and keep flatmem the default.
> > 
> > Enabling sparsemem (with SPARSEMEM_STATIC) for defconfig build increases
> > the kernel nearly by 8k and most probably will make things slower.
> 
> I think when selecting the default, you need to consider whether
> platforms can boot with that default configuration or not.
> 
> There are cases where the reason to use sparsemem is not an optional
> one but is out of necessity - they require the page array to be split
> up in order to boot successfully.
> 
> With that in mind, flatmem becomes an "optimisation" over sparsemem.

At the moment, there are three platforms that enable SPARSEMEM: ARCH_EP93XX, 
ARCH_RPC and ARCH_SA1100. All the rest have FLATMEM implcitly selected.

I do not intend to change that, I am only going add an ability to select
the memory model for ARCH_MULTIPLATFORM. 

I'll respin the series on the list before adding the patches to the
patch system.

> -- 
> RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
> FTTC for 0.8m (est. 1762m) line in suburbia: sync at 13.1Mbps down 424kbps up

Arnd Bergmann May 19, 2020, 3:32 p.m. UTC | #9

On Tue, May 19, 2020 at 5:27 PM Mike Rapoport <[e-mail]> wrote:
> On Tue, May 19, 2020 at 04:04:52PM +0100, Russell King - ARM Linux admin wrote:
> > There are cases where the reason to use sparsemem is not an optional
> > one but is out of necessity - they require the page array to be split
> > up in order to boot successfully.
> >
> > With that in mind, flatmem becomes an "optimisation" over sparsemem.
>
> At the moment, there are three platforms that enable SPARSEMEM: ARCH_EP93XX,
> ARCH_RPC and ARCH_SA1100. All the rest have FLATMEM implcitly selected.
>
> I do not intend to change that, I am only going add an ability to select
> the memory model for ARCH_MULTIPLATFORM.
>
> I'll respin the series on the list before adding the patches to the
> patch system.

I think we'll make EP93xx part of multiplatform at some point, IIRC
only the missing clock driver is stopping us at the moment, and I already
discussed with Linus Walleij how that can be done.

My guess is that ep93xx is one platform on which sparsemem is
just an optimization to reduce the initial memory consumption, but
we should verify that when we get there.

      Arnd

Russell King (Oracle) May 19, 2020, 4:54 p.m. UTC | #10

On Tue, May 19, 2020 at 05:32:52PM +0200, Arnd Bergmann wrote:
> On Tue, May 19, 2020 at 5:27 PM Mike Rapoport <[e-mail]> wrote:
> > On Tue, May 19, 2020 at 04:04:52PM +0100, Russell King - ARM Linux admin wrote:
> > > There are cases where the reason to use sparsemem is not an optional
> > > one but is out of necessity - they require the page array to be split
> > > up in order to boot successfully.
> > >
> > > With that in mind, flatmem becomes an "optimisation" over sparsemem.
> >
> > At the moment, there are three platforms that enable SPARSEMEM: ARCH_EP93XX,
> > ARCH_RPC and ARCH_SA1100. All the rest have FLATMEM implcitly selected.
> >
> > I do not intend to change that, I am only going add an ability to select
> > the memory model for ARCH_MULTIPLATFORM.
> >
> > I'll respin the series on the list before adding the patches to the
> > patch system.
> 
> I think we'll make EP93xx part of multiplatform at some point, IIRC
> only the missing clock driver is stopping us at the moment, and I already
> discussed with Linus Walleij how that can be done.
> 
> My guess is that ep93xx is one platform on which sparsemem is
> just an optimization to reduce the initial memory consumption, but
> we should verify that when we get there.

When you have a platform where the memory is segmented into separate
blocks widely spaced, sparsemem is not an optimisation.  For example,
If you have four blocks spaced across 1GB, that requires about
256Ki struct page's.  Assuming 32 byte struct page, that requires 8MiB
of contiguous memory.

If we also consider the other constraint - that the kernel has to fit
in the first bank of memory, then, considering the size of the kernel
(the first two are non-multiplatform kernels):

text     data   bss      dec      hex      filename
4045505  903700   92984  5042189  4cf00d   n2100/boot/vmlinux-5.6.12+
4045361  957276 1159052  6161689  5e0519   assabet/boot/vmlinux-5.2.0+
6933363 1451420  203984  8588767  830ddf   virt/boot/vmlinux-5.6.0+
9980260 3568070 7403296 20951626 13fb24a   multi/boot/vmlinux-5.3.0+

So, realistically, we're looking at imposing a requirement that the
first bank of memory is no smaller than 16MB on these machines if a
"default" flatmem multiplatform kernel is going to be able to boot,
and if all banks are populated, that there is another bank that has
at least 8MB to hold the memmap array.

BTW, the "optimisation" argument for sparsemem doesn't actually
hold.  For flatmem, we free the unused parts of the memmap array,
freeing those pages for other uses.  Sparsemem on ARM is about
allowing these platforms to boot with their memory spread across
the physical address space savannah.

Mike Rapoport May 19, 2020, 5:59 p.m. UTC | #11

Hi Russell,

On Tue, May 19, 2020 at 05:54:45PM +0100, Russell King - ARM Linux admin wrote:
> On Tue, May 19, 2020 at 05:32:52PM +0200, Arnd Bergmann wrote:
> > On Tue, May 19, 2020 at 5:27 PM Mike Rapoport <[e-mail]> wrote:
> > > On Tue, May 19, 2020 at 04:04:52PM +0100, Russell King - ARM Linux admin wrote:
> > > > There are cases where the reason to use sparsemem is not an optional
> > > > one but is out of necessity - they require the page array to be split
> > > > up in order to boot successfully.
> > > >
> > > > With that in mind, flatmem becomes an "optimisation" over sparsemem.
> > >
> > > At the moment, there are three platforms that enable SPARSEMEM: ARCH_EP93XX,
> > > ARCH_RPC and ARCH_SA1100. All the rest have FLATMEM implcitly selected.
> > >
> > > I do not intend to change that, I am only going add an ability to select
> > > the memory model for ARCH_MULTIPLATFORM.
> > >
> > > I'll respin the series on the list before adding the patches to the
> > > patch system.
> > 
> > I think we'll make EP93xx part of multiplatform at some point, IIRC
> > only the missing clock driver is stopping us at the moment, and I already
> > discussed with Linus Walleij how that can be done.
> > 
> > My guess is that ep93xx is one platform on which sparsemem is
> > just an optimization to reduce the initial memory consumption, but
> > we should verify that when we get there.
> 
> When you have a platform where the memory is segmented into separate
> blocks widely spaced, sparsemem is not an optimisation.  For example,
> If you have four blocks spaced across 1GB, that requires about
> 256Ki struct page's.  Assuming 32 byte struct page, that requires 8MiB
> of contiguous memory.
> 
> If we also consider the other constraint - that the kernel has to fit
> in the first bank of memory, then, considering the size of the kernel
> (the first two are non-multiplatform kernels):
> 
> text     data   bss      dec      hex      filename
> 4045505  903700   92984  5042189  4cf00d   n2100/boot/vmlinux-5.6.12+
> 4045361  957276 1159052  6161689  5e0519   assabet/boot/vmlinux-5.2.0+
> 6933363 1451420  203984  8588767  830ddf   virt/boot/vmlinux-5.6.0+
> 9980260 3568070 7403296 20951626 13fb24a   multi/boot/vmlinux-5.3.0+
> 
> So, realistically, we're looking at imposing a requirement that the
> first bank of memory is no smaller than 16MB on these machines if a
> "default" flatmem multiplatform kernel is going to be able to boot,
> and if all banks are populated, that there is another bank that has
> at least 8MB to hold the memmap array.
> 
> BTW, the "optimisation" argument for sparsemem doesn't actually
> hold.  For flatmem, we free the unused parts of the memmap array,
> freeing those pages for other uses.  Sparsemem on ARM is about
> allowing these platforms to boot with their memory spread across
> the physical address space savannah.

There is no argument that sparsemem is more robust and flexible in its
ability to hande separated memory banks. And, AFAIU, nobody suggests to
start using flatmem instead of sparsemem on any machine.

Currently, when you build the kernel with "make defconfig && make" or
"make multi_X_defconfig && make" kconfig implicitly selects flatmem and
there is no possiblity to enable sparsemem in the menuconfig.

Florian sent the pathces with the purpose to allow overriding the
explict choise of flatmem with sparsemem in menuconfig.

At some point I've asked whether we would like to use sparsemem instead
of flatmem as the default for the multiplafrom build. The upside of such
change is the robustness you described above. The downside is a bit
larger kernel image and slight slow down in page_to_pfn and pfn_to_page.

That said we have two options:
* Massage the patches Florian sent and add ability to select memory
model in menuconfig for the multiplatform builds without changing
anything in configurations that were not manually atered.
* Simply add 'select ARCH_SPAPSEMEM_ENABLE' under 'ARCH_MULTIPLAFROM'
and force all multiplatform builds to use sparsemem.

So the question is what is the preferred way to move forward.

> -- 
> RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
> FTTC for 0.8m (est. 1762m) line in suburbia: sync at 13.1Mbps down 424kbps up

Arnd Bergmann May 19, 2020, 8:42 p.m. UTC | #12

On Tue, May 19, 2020 at 6:54 PM Russell King - ARM Linux admin
<[e-mail]> wrote:
> On Tue, May 19, 2020 at 05:32:52PM +0200, Arnd Bergmann wrote:
> > On Tue, May 19, 2020 at 5:27 PM Mike Rapoport <[e-mail]> wrote:
> > > On Tue, May 19, 2020 at 04:04:52PM +0100, Russell King - ARM Linux admin wrote:
> > > > There are cases where the reason to use sparsemem is not an optional
> > > > one but is out of necessity - they require the page array to be split
> > > > up in order to boot successfully.
> > > >
> > > > With that in mind, flatmem becomes an "optimisation" over sparsemem.
> > >
> > > At the moment, there are three platforms that enable SPARSEMEM: ARCH_EP93XX,
> > > ARCH_RPC and ARCH_SA1100. All the rest have FLATMEM implcitly selected.
> > >
> > > I do not intend to change that, I am only going add an ability to select
> > > the memory model for ARCH_MULTIPLATFORM.
> > >
> > > I'll respin the series on the list before adding the patches to the
> > > patch system.
> >
> > I think we'll make EP93xx part of multiplatform at some point, IIRC
> > only the missing clock driver is stopping us at the moment, and I already
> > discussed with Linus Walleij how that can be done.
> >
> > My guess is that ep93xx is one platform on which sparsemem is
> > just an optimization to reduce the initial memory consumption, but
> > we should verify that when we get there.
>
> When you have a platform where the memory is segmented into separate
> blocks widely spaced, sparsemem is not an optimisation.  For example,
> If you have four blocks spaced across 1GB, that requires about
> 256Ki struct page's.  Assuming 32 byte struct page, that requires 8MiB
> of contiguous memory.

I checked the datasheet: ep93xx has four chip-selects, each of them
with their own 256MB memory area. Most boards we support use only
the last chipselect, but those that have more than one would waste
up to 2MB per additional memory chip, which is not nice but doesn't
make it a hard requirement as long as the kernel can fit into the first one.

However, the chips has a second problem: depending on the
specific memory configuration, only every other 2/4/8MB within
that range may actually be used, so the mem_map[] array would
double its size in practice and (worse) the kernel would likely
not fit into the first section without sparsemem.

This means we should indeed treat it as a requirement for ep93xx.

     Arnd

Florian Fainelli May 21, 2020, 2:45 a.m. UTC | #13

On 5/19/2020 1:42 PM, Arnd Bergmann wrote:
> On Tue, May 19, 2020 at 6:54 PM Russell King - ARM Linux admin
> <[e-mail]> wrote:
>> On Tue, May 19, 2020 at 05:32:52PM +0200, Arnd Bergmann wrote:
>>> On Tue, May 19, 2020 at 5:27 PM Mike Rapoport <[e-mail]> wrote:
>>>> On Tue, May 19, 2020 at 04:04:52PM +0100, Russell King - ARM Linux admin wrote:
>>>>> There are cases where the reason to use sparsemem is not an optional
>>>>> one but is out of necessity - they require the page array to be split
>>>>> up in order to boot successfully.
>>>>>
>>>>> With that in mind, flatmem becomes an "optimisation" over sparsemem.
>>>>
>>>> At the moment, there are three platforms that enable SPARSEMEM: ARCH_EP93XX,
>>>> ARCH_RPC and ARCH_SA1100. All the rest have FLATMEM implcitly selected.
>>>>
>>>> I do not intend to change that, I am only going add an ability to select
>>>> the memory model for ARCH_MULTIPLATFORM.
>>>>
>>>> I'll respin the series on the list before adding the patches to the
>>>> patch system.
>>>
>>> I think we'll make EP93xx part of multiplatform at some point, IIRC
>>> only the missing clock driver is stopping us at the moment, and I already
>>> discussed with Linus Walleij how that can be done.
>>>
>>> My guess is that ep93xx is one platform on which sparsemem is
>>> just an optimization to reduce the initial memory consumption, but
>>> we should verify that when we get there.
>>
>> When you have a platform where the memory is segmented into separate
>> blocks widely spaced, sparsemem is not an optimisation.  For example,
>> If you have four blocks spaced across 1GB, that requires about
>> 256Ki struct page's.  Assuming 32 byte struct page, that requires 8MiB
>> of contiguous memory.
> 
> I checked the datasheet: ep93xx has four chip-selects, each of them
> with their own 256MB memory area. Most boards we support use only
> the last chipselect, but those that have more than one would waste
> up to 2MB per additional memory chip, which is not nice but doesn't
> make it a hard requirement as long as the kernel can fit into the first one.
> 
> However, the chips has a second problem: depending on the
> specific memory configuration, only every other 2/4/8MB within
> that range may actually be used, so the mem_map[] array would
> double its size in practice and (worse) the kernel would likely
> not fit into the first section without sparsemem.
> 
> This means we should indeed treat it as a requirement for ep93xx.

That is already captured with the select ARCH_SPARSEMEM_ENABLE or the
ARCH_EP93XX entry.

Should we just make it possible for multi platform users to override the
memory model in their configuration file and later we just default
ARCH_MULTI_V7 to sparsemem?

Arnd Bergmann May 21, 2020, 7:47 a.m. UTC | #14

On Thu, May 21, 2020 at 4:45 AM Florian Fainelli <[e-mail]> wrote:
> On 5/19/2020 1:42 PM, Arnd Bergmann wrote:
> > On Tue, May 19, 2020 at 6:54 PM Russell King - ARM Linux admin <[e-mail]> wrote:
> >
> > I checked the datasheet: ep93xx has four chip-selects, each of them
> > with their own 256MB memory area. Most boards we support use only
> > the last chipselect, but those that have more than one would waste
> > up to 2MB per additional memory chip, which is not nice but doesn't
> > make it a hard requirement as long as the kernel can fit into the first one.
> >
> > However, the chips has a second problem: depending on the
> > specific memory configuration, only every other 2/4/8MB within
> > that range may actually be used, so the mem_map[] array would
> > double its size in practice and (worse) the kernel would likely
> > not fit into the first section without sparsemem.
> >
> > This means we should indeed treat it as a requirement for ep93xx.
>
> That is already captured with the select ARCH_SPARSEMEM_ENABLE or the
> ARCH_EP93XX entry.

Yes, we just need to keep it that way when ep93xx eventually
becomes part of the ARCH_MULTI_V4T configuration.

> Should we just make it possible for multi platform users to override the
> memory model in their configuration file and later we just default
> ARCH_MULTI_V7 to sparsemem?

I think there are three steps:

1. make the memory model user-selectable for all multiplatform kernels
   to allow sparsemem
2. change the defconfig files for platforms that would likely benefit
  (multi_v7_defconfig, exynos_defconfig, imx_v6_v7_defconfig, ...)
3. change the default for ARCH_MULTI_V7, or alternatively for
   configurations that include platforms that usually want it.

My preference would be to have the first two in v5.8, but leave
the third one for a later time, depending on who needs it.
I notice that the changelog text does not give a particular
example of platforms with very sparse memory maps. It
might help to have more background there. Among the platforms
I see, it seems fairly rare to have any RAM above the 4GB
boundary on ARMv7 based chips: keystone2, r-car, and
highbank would be the main examples I can think of, but I
don't know if they have large holes in the memory map.

       Arnd

Patch

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 5986277296c3..7bb5264a9c3a 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -312,6 +312,8 @@  choice
 config ARCH_MULTIPLATFORM
 	bool "Allow multiple platforms to be selected"
 	depends on MMU
+	select ARCH_FLATMEM_ENABLE
+	select ARCH_SPARSEMEM_ENABLE
 	select ARM_HAS_SG_CHAIN
 	select ARM_PATCH_PHYS_VIRT
 	select AUTO_ZRELADDR
@@ -1515,6 +1517,9 @@  config OABI_COMPAT
 config ARCH_HAS_HOLES_MEMORYMODEL
 	bool
 
+config ARCH_FLATMEM_ENABLE
+	bool
+
 config ARCH_SPARSEMEM_ENABLE
 	bool