| Message ID | 20201215152918.1511108-1-maz@kernel.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | arm64: Warn the user when a small VA_BITS value wastes memory | expand |
On Tue, Dec 15, 2020 at 03:29:18PM +0000, Marc Zyngier wrote: > The memblock code ignores any memory that doesn't fit in the > linear mapping. In order to preserve the distance between two physical > memory locations and their mappings in the linear map, any hole between > two memory regions occupies the same space in the linear map. > > On most systems, this is hardly a problem (the memory banks are close > together, and VA_BITS represents a large space compared to the available > memory *and* the potential gaps). > > On NUMA systems, things are quite different: the gaps between the > memory nodes can be pretty large compared to the memory size itself, > and the range from memblock_start_of_DRAM() to memblock_end_of_DRAM() > can exceed the space described by VA_BITS. > > Unfortunately, we're not very good at making this obvious to the user, > and on a D05 system (two sockets and 4 nodes with 64GB each) > accidentally configured with 39bit VA, we display something like this: > > [ 0.000000] NUMA: NODE_DATA [mem 0x1ffbffe100-0x1ffbffffff] > [ 0.000000] NUMA: NODE_DATA [mem 0x2febfc1100-0x2febfc2fff] > [ 0.000000] NUMA: Initmem setup node 2 [<memory-less node>] > [ 0.000000] NUMA: NODE_DATA [mem 0x2febfbf200-0x2febfc10ff] > [ 0.000000] NUMA: NODE_DATA(2) on node 1 > [ 0.000000] NUMA: Initmem setup node 3 [<memory-less node>] > [ 0.000000] NUMA: NODE_DATA [mem 0x2febfbd300-0x2febfbf1ff] > [ 0.000000] NUMA: NODE_DATA(3) on node 1 > > which isn't very explicit, and doesn't tell the user why 128GB > have suddently disappeared. > > Let's add a warning message telling the user that memory has been > truncated, and offer a potential solution (bumping VA_BITS up). > > Signed-off-by: Marc Zyngier <maz@kernel.org> > --- > arch/arm64/mm/init.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c > index 095540667f0f..cd5da059de85 100644 > --- a/arch/arm64/mm/init.c > +++ b/arch/arm64/mm/init.c > @@ -283,6 +283,9 @@ void __init arm64_memblock_init(void) > memstart_addr = round_down(memblock_start_of_DRAM(), > ARM64_MEMSTART_ALIGN); > > + if ((memblock_end_of_DRAM() - memstart_addr) > linear_region_size) > + pr_warn("Memory doesn't fit in the linear mapping, VA_BITS too small\n"); Hmm... we might want to restructure this code a bit, since we have a few checks that are /almost/ the same, and we could probably place all the memory clipping under this new condition. That said, after having a go, I couldn't find a neat solution. FWIW, for this as-is: Acked-by: Mark Rutland <mark.rutland@arm.com> Mark.
On Tue, 15 Dec 2020 15:29:18 +0000, Marc Zyngier wrote: > The memblock code ignores any memory that doesn't fit in the > linear mapping. In order to preserve the distance between two physical > memory locations and their mappings in the linear map, any hole between > two memory regions occupies the same space in the linear map. > > On most systems, this is hardly a problem (the memory banks are close > together, and VA_BITS represents a large space compared to the available > memory *and* the potential gaps). > > [...] Applied to arm64 (for-next/fixes), thanks! [1/1] arm64: Warn the user when a small VA_BITS value wastes memory https://git.kernel.org/arm64/c/31f80a4e9603
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 095540667f0f..cd5da059de85 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -283,6 +283,9 @@ void __init arm64_memblock_init(void) memstart_addr = round_down(memblock_start_of_DRAM(), ARM64_MEMSTART_ALIGN); + if ((memblock_end_of_DRAM() - memstart_addr) > linear_region_size) + pr_warn("Memory doesn't fit in the linear mapping, VA_BITS too small\n"); + /* * Remove the memory that we will not be able to cover with the * linear mapping. Take care not to clip the kernel which may be
The memblock code ignores any memory that doesn't fit in the linear mapping. In order to preserve the distance between two physical memory locations and their mappings in the linear map, any hole between two memory regions occupies the same space in the linear map. On most systems, this is hardly a problem (the memory banks are close together, and VA_BITS represents a large space compared to the available memory *and* the potential gaps). On NUMA systems, things are quite different: the gaps between the memory nodes can be pretty large compared to the memory size itself, and the range from memblock_start_of_DRAM() to memblock_end_of_DRAM() can exceed the space described by VA_BITS. Unfortunately, we're not very good at making this obvious to the user, and on a D05 system (two sockets and 4 nodes with 64GB each) accidentally configured with 39bit VA, we display something like this: [ 0.000000] NUMA: NODE_DATA [mem 0x1ffbffe100-0x1ffbffffff] [ 0.000000] NUMA: NODE_DATA [mem 0x2febfc1100-0x2febfc2fff] [ 0.000000] NUMA: Initmem setup node 2 [<memory-less node>] [ 0.000000] NUMA: NODE_DATA [mem 0x2febfbf200-0x2febfc10ff] [ 0.000000] NUMA: NODE_DATA(2) on node 1 [ 0.000000] NUMA: Initmem setup node 3 [<memory-less node>] [ 0.000000] NUMA: NODE_DATA [mem 0x2febfbd300-0x2febfbf1ff] [ 0.000000] NUMA: NODE_DATA(3) on node 1 which isn't very explicit, and doesn't tell the user why 128GB have suddently disappeared. Let's add a warning message telling the user that memory has been truncated, and offer a potential solution (bumping VA_BITS up). Signed-off-by: Marc Zyngier <maz@kernel.org> --- arch/arm64/mm/init.c | 3 +++ 1 file changed, 3 insertions(+)