| Message ID | 20240703210707.1986816-1-scott@os.amperecomputing.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | arm64: limit MAX_PHYSMEM_BITS based on vmemmap | expand |
Hello Scott, On 7/4/24 02:37, D Scott Phillips wrote: > Prior to the memory map adjustments in v6.9-rc1, the amdgpu driver could > trip over the warning of: > > `WARN_ON((start < VMEMMAP_START) || (end > VMEMMAP_END));` Could you please provide the mainline commit ID for the mentioned memory adjustment changes. > > in vmemmap_populate()[1]. After the adjustments, it becomes a translation > fault and panic. Probably caused by accessing memory which does not exist on that address. > > The cause is that the amdgpu driver allocates some unused space from > iomem_resource and claims it as MEMORY_DEVICE_PRIVATE and > devm_memremap_pages() it. An address above those backed by the vmemmap is > used. > > Adjust MAX_PHYSMEM_BITS so that addresses not backed by the vmemmap will > not be chosen as device private addresses. > > [1]: Call trace: > vmemmap_populate+0x30/0x48 > __populate_section_memmap+0x40/0x90 > sparse_add_section+0xfc/0x3e8 > __add_pages+0xb4/0x168 > pagemap_range+0x300/0x410 > memremap_pages+0x184/0x2d8 > devm_memremap_pages+0x30/0x90 > kgd2kfd_init_zone_device+0xe0/0x1f0 [amdgpu] > amdgpu_device_ip_init+0x674/0x888 [amdgpu] > amdgpu_device_init+0x7bc/0xed8 [amdgpu] > amdgpu_driver_load_kms+0x28/0x1c0 [amdgpu] > amdgpu_pci_probe+0x194/0x580 [amdgpu] > local_pci_probe+0x48/0xb8 > work_for_cpu_fn+0x24/0x40 > process_one_work+0x170/0x3e0 > worker_thread+0x2ac/0x3e0 > kthread+0xf4/0x108 > ret_from_fork+0x10/0x20 > > Signed-off-by: D Scott Phillips <scott@os.amperecomputing.com> > --- > arch/arm64/include/asm/sparsemem.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h > index 8a8acc220371c..8387301f2e206 100644 > --- a/arch/arm64/include/asm/sparsemem.h > +++ b/arch/arm64/include/asm/sparsemem.h > @@ -5,7 +5,7 @@ > #ifndef __ASM_SPARSEMEM_H > #define __ASM_SPARSEMEM_H > > -#define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS > +#define MAX_PHYSMEM_BITS ilog2(VMEMMAP_RANGE) Just wondering if there is another method, which avoids selecting physical memory ranges not backed with vmemmap. Also will reducing MAX_PHYSMEM_BITS below ARM64_PA_BITS have other side effects ? Do other platforms have this exact same co-relation between MAX_PHYSMEM_BITS and vmemmap range ? > > /* > * Section size must be at least 512MB for 64K base
On Thu, Jul 04, 2024 at 08:42:52AM +0530, Anshuman Khandual wrote: > On 7/4/24 02:37, D Scott Phillips wrote: > > diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h > > index 8a8acc220371c..8387301f2e206 100644 > > --- a/arch/arm64/include/asm/sparsemem.h > > +++ b/arch/arm64/include/asm/sparsemem.h > > @@ -5,7 +5,7 @@ > > #ifndef __ASM_SPARSEMEM_H > > #define __ASM_SPARSEMEM_H > > > > -#define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS > > +#define MAX_PHYSMEM_BITS ilog2(VMEMMAP_RANGE) > > Just wondering if there is another method, which avoids selecting physical > memory ranges not backed with vmemmap. Also will reducing MAX_PHYSMEM_BITS > below ARM64_PA_BITS have other side effects ? Do other platforms have this > exact same co-relation between MAX_PHYSMEM_BITS and vmemmap range ? That's indeed a pretty weird workaround. MAX_PHYSMEM_BITS, as the name implies, is about the physical bits supported for memory while VMEMMAP_RANGE tells us the virtual address range. There is a correlation between them but they are different things conceptually. The memory hotplug code uses arch_get_mappable_range(). This should be called from the amdgpu code rather than changing MAX_PHYSMEM_BITS.
diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h index 8a8acc220371c..8387301f2e206 100644 --- a/arch/arm64/include/asm/sparsemem.h +++ b/arch/arm64/include/asm/sparsemem.h @@ -5,7 +5,7 @@ #ifndef __ASM_SPARSEMEM_H #define __ASM_SPARSEMEM_H -#define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS +#define MAX_PHYSMEM_BITS ilog2(VMEMMAP_RANGE) /* * Section size must be at least 512MB for 64K base
Prior to the memory map adjustments in v6.9-rc1, the amdgpu driver could trip over the warning of: `WARN_ON((start < VMEMMAP_START) || (end > VMEMMAP_END));` in vmemmap_populate()[1]. After the adjustments, it becomes a translation fault and panic. The cause is that the amdgpu driver allocates some unused space from iomem_resource and claims it as MEMORY_DEVICE_PRIVATE and devm_memremap_pages() it. An address above those backed by the vmemmap is used. Adjust MAX_PHYSMEM_BITS so that addresses not backed by the vmemmap will not be chosen as device private addresses. [1]: Call trace: vmemmap_populate+0x30/0x48 __populate_section_memmap+0x40/0x90 sparse_add_section+0xfc/0x3e8 __add_pages+0xb4/0x168 pagemap_range+0x300/0x410 memremap_pages+0x184/0x2d8 devm_memremap_pages+0x30/0x90 kgd2kfd_init_zone_device+0xe0/0x1f0 [amdgpu] amdgpu_device_ip_init+0x674/0x888 [amdgpu] amdgpu_device_init+0x7bc/0xed8 [amdgpu] amdgpu_driver_load_kms+0x28/0x1c0 [amdgpu] amdgpu_pci_probe+0x194/0x580 [amdgpu] local_pci_probe+0x48/0xb8 work_for_cpu_fn+0x24/0x40 process_one_work+0x170/0x3e0 worker_thread+0x2ac/0x3e0 kthread+0xf4/0x108 ret_from_fork+0x10/0x20 Signed-off-by: D Scott Phillips <scott@os.amperecomputing.com> --- arch/arm64/include/asm/sparsemem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)