powerpc: ensure that swiotlb buffer is allocated from low memory

Commit Message

Mike Rapoport Dec. 4, 2019, 12:35 p.m. UTC

From: Mike Rapoport <[e-mail]>

Some powerpc platforms (e.g. 85xx) limit DMA-able memory way below 4G. If a
system has more physical memory than this limit, the swiotlb buffer is not
addressable because it is allocated from memblock using top-down mode.

Force memblock to bottom-up mode before calling swiotlb_init() to ensure
that the swiotlb buffer is DMA-able.

Link: https://lkml.kernel.org/r/[e-mail]
Reported-by: Christian Zigotzky <[e-mail]>
Signed-off-by: Mike Rapoport <[e-mail]>
Cc: Benjamin Herrenschmidt <[e-mail]>
Cc: Christoph Hellwig <[e-mail]>
Cc: Darren Stevens <[e-mail]>
Cc: mad skateman <[e-mail]>
Cc: Michael Ellerman <[e-mail]>
Cc: Nicolas Saenz Julienne <[e-mail]>
Cc: Paul Mackerras <[e-mail]>
Cc: Robin Murphy <[e-mail]>
Cc: Rob Herring <[e-mail]>
---
 arch/powerpc/mm/mem.c | 8 ++++++++
 1 file changed, 8 insertions(+)

Comments

Christoph Hellwig Dec. 4, 2019, 1:56 p.m. UTC | #1

On Wed, Dec 04, 2019 at 02:35:24PM +0200, Mike Rapoport wrote:
> From: Mike Rapoport <[e-mail]>
> 
> Some powerpc platforms (e.g. 85xx) limit DMA-able memory way below 4G. If a
> system has more physical memory than this limit, the swiotlb buffer is not
> addressable because it is allocated from memblock using top-down mode.
> 
> Force memblock to bottom-up mode before calling swiotlb_init() to ensure
> that the swiotlb buffer is DMA-able.
> 
> Link: https://lkml.kernel.org/r/[e-mail]
> Reported-by: Christian Zigotzky <[e-mail]>
> Signed-off-by: Mike Rapoport <[e-mail]>

Looks good:

Reviewed-by: Christoph Hellwig <[e-mail]>

Michael Ellerman Dec. 9, 2019, 5:43 a.m. UTC | #2

Mike Rapoport <[e-mail]> writes:
> From: Mike Rapoport <[e-mail]>
>
> Some powerpc platforms (e.g. 85xx) limit DMA-able memory way below 4G. If a
> system has more physical memory than this limit, the swiotlb buffer is not
> addressable because it is allocated from memblock using top-down mode.
>
> Force memblock to bottom-up mode before calling swiotlb_init() to ensure
> that the swiotlb buffer is DMA-able.
>
> Link: https://lkml.kernel.org/r/[e-mail]

This wasn't bisected, but I thought it was a regression. Do we know what
commit caused it?

Was it 25078dc1f74b ("powerpc: use mm zones more sensibly") ?

Or was that a red herring?

cheers

> Reported-by: Christian Zigotzky <[e-mail]>
> Signed-off-by: Mike Rapoport <[e-mail]>
> Cc: Benjamin Herrenschmidt <[e-mail]>
> Cc: Christoph Hellwig <[e-mail]>
> Cc: Darren Stevens <[e-mail]>
> Cc: mad skateman <[e-mail]>
> Cc: Michael Ellerman <[e-mail]>
> Cc: Nicolas Saenz Julienne <[e-mail]>
> Cc: Paul Mackerras <[e-mail]>
> Cc: Robin Murphy <[e-mail]>
> Cc: Rob Herring <[e-mail]>
> ---
>  arch/powerpc/mm/mem.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
> index be941d382c8d..14c2c53e3f9e 100644
> --- a/arch/powerpc/mm/mem.c
> +++ b/arch/powerpc/mm/mem.c
> @@ -260,6 +260,14 @@ void __init mem_init(void)
>  	BUILD_BUG_ON(MMU_PAGE_COUNT > 16);
>  
>  #ifdef CONFIG_SWIOTLB
> +	/*
> +	 * Some platforms (e.g. 85xx) limit DMA-able memory way below
> +	 * 4G. We force memblock to bottom-up mode to ensure that the
> +	 * memory allocated in swiotlb_init() is DMA-able.
> +	 * As it's the last memblock allocation, no need to reset it
> +	 * back to to-down.
> +	 */
> +	memblock_set_bottom_up(true);
>  	swiotlb_init(0);
>  #endif
>  
> -- 
> 2.24.0

Mike Rapoport Dec. 9, 2019, 7:54 a.m. UTC | #3

On Mon, Dec 09, 2019 at 04:43:17PM +1100, Michael Ellerman wrote:
> Mike Rapoport <[e-mail]> writes:
> > From: Mike Rapoport <[e-mail]>
> >
> > Some powerpc platforms (e.g. 85xx) limit DMA-able memory way below 4G. If a
> > system has more physical memory than this limit, the swiotlb buffer is not
> > addressable because it is allocated from memblock using top-down mode.
> >
> > Force memblock to bottom-up mode before calling swiotlb_init() to ensure
> > that the swiotlb buffer is DMA-able.
> >
> > Link: https://lkml.kernel.org/r/[e-mail]
> 
> This wasn't bisected, but I thought it was a regression. Do we know what
> commit caused it?
> 
> Was it 25078dc1f74b ("powerpc: use mm zones more sensibly") ?

swiotlb buffer is initialized before zones are actually used, so probably
not :)
 
> Or was that a red herring?
> 
> cheers
> 
> > Reported-by: Christian Zigotzky <[e-mail]>
> > Signed-off-by: Mike Rapoport <[e-mail]>
> > Cc: Benjamin Herrenschmidt <[e-mail]>
> > Cc: Christoph Hellwig <[e-mail]>
> > Cc: Darren Stevens <[e-mail]>
> > Cc: mad skateman <[e-mail]>
> > Cc: Michael Ellerman <[e-mail]>
> > Cc: Nicolas Saenz Julienne <[e-mail]>
> > Cc: Paul Mackerras <[e-mail]>
> > Cc: Robin Murphy <[e-mail]>
> > Cc: Rob Herring <[e-mail]>
> > ---
> >  arch/powerpc/mm/mem.c | 8 ++++++++
> >  1 file changed, 8 insertions(+)
> >
> > diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
> > index be941d382c8d..14c2c53e3f9e 100644
> > --- a/arch/powerpc/mm/mem.c
> > +++ b/arch/powerpc/mm/mem.c
> > @@ -260,6 +260,14 @@ void __init mem_init(void)
> >  	BUILD_BUG_ON(MMU_PAGE_COUNT > 16);
> >  
> >  #ifdef CONFIG_SWIOTLB
> > +	/*
> > +	 * Some platforms (e.g. 85xx) limit DMA-able memory way below
> > +	 * 4G. We force memblock to bottom-up mode to ensure that the
> > +	 * memory allocated in swiotlb_init() is DMA-able.
> > +	 * As it's the last memblock allocation, no need to reset it
> > +	 * back to to-down.
> > +	 */
> > +	memblock_set_bottom_up(true);
> >  	swiotlb_init(0);
> >  #endif
> >  
> > -- 
> > 2.24.0

Michael Ellerman Dec. 18, 2019, 4:05 a.m. UTC | #4

On Wed, 2019-12-04 at 12:35:24 UTC, Mike Rapoport wrote:
> From: Mike Rapoport <[e-mail]>
> 
> Some powerpc platforms (e.g. 85xx) limit DMA-able memory way below 4G. If a
> system has more physical memory than this limit, the swiotlb buffer is not
> addressable because it is allocated from memblock using top-down mode.
> 
> Force memblock to bottom-up mode before calling swiotlb_init() to ensure
> that the swiotlb buffer is DMA-able.
> 
> Link: https://lkml.kernel.org/r/[e-mail]
> Reported-by: Christian Zigotzky <[e-mail]>
> Signed-off-by: Mike Rapoport <[e-mail]>

Applied to powerpc fixes, thanks.

https://git.kernel.org/powerpc/c/8fabc623238e68b3ac63c0dd1657bf86c1fa33af

cheers

Patch

diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index be941d382c8d..14c2c53e3f9e 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -260,6 +260,14 @@  void __init mem_init(void)
 	BUILD_BUG_ON(MMU_PAGE_COUNT > 16);
 
 #ifdef CONFIG_SWIOTLB
+	/*
+	 * Some platforms (e.g. 85xx) limit DMA-able memory way below
+	 * 4G. We force memblock to bottom-up mode to ensure that the
+	 * memory allocated in swiotlb_init() is DMA-able.
+	 * As it's the last memblock allocation, no need to reset it
+	 * back to to-down.
+	 */
+	memblock_set_bottom_up(true);
 	swiotlb_init(0);
 #endif