| Message ID | 20220209092642.9181-1-shijie@os.amperecomputing.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | arm64: crash_core: Export MODULES, VMALLOC, and VMEMMAP ranges | expand |
On 02/09/22 at 09:26am, Huang Shijie wrote: > The following interrelated ranges are needed by the kdump crash tool: > MODULES_VADDR ~ MODULES_END, > VMALLOC_START ~ VMALLOC_END, > VMEMMAP_START ~ VMEMMAP_END > > Since these values change from time to time, it is preferable to export > them via vmcoreinfo than to change the crash's code frequently. Add Kazu and Lianbo to CC since they take care of Crash utility now, please check if this is necessary for Crash. > > Signed-off-by: Huang Shijie <shijie@os.amperecomputing.com> > --- > Documentation/admin-guide/kdump/vmcoreinfo.rst | 8 ++++++++ > arch/arm64/kernel/crash_core.c | 6 ++++++ > 2 files changed, 14 insertions(+) > > diff --git a/Documentation/admin-guide/kdump/vmcoreinfo.rst b/Documentation/admin-guide/kdump/vmcoreinfo.rst > index 3861a25faae1..a339af45a22e 100644 > --- a/Documentation/admin-guide/kdump/vmcoreinfo.rst > +++ b/Documentation/admin-guide/kdump/vmcoreinfo.rst > @@ -494,6 +494,14 @@ architecture which is used to lookup the page-tables for the Virtual > addresses in the higher VA range (refer to ARMv8 ARM document for > more details). > > +MODULES_VADDR|MODULES_END|VMALLOC_START|VMALLOC_END|VMEMMAP_START|VMEMMAP_END > +------------- > + > +Used to get the correct ranges: > + MODULES_VADDR ~ MODULES_END-1 : Kernel module space. > + VMALLOC_START ~ VMALLOC_END-1 : vmalloc() / ioremap() space. > + VMEMMAP_START ~ VMEMMAP_END-1 : vmemmap region, used for struct page array. > + > arm > === > > diff --git a/arch/arm64/kernel/crash_core.c b/arch/arm64/kernel/crash_core.c > index 314391a156ee..2b65aae332ce 100644 > --- a/arch/arm64/kernel/crash_core.c > +++ b/arch/arm64/kernel/crash_core.c > @@ -20,6 +20,12 @@ void arch_crash_save_vmcoreinfo(void) > { > VMCOREINFO_NUMBER(VA_BITS); > /* Please note VMCOREINFO_NUMBER() uses "%d", not "%x" */ > + vmcoreinfo_append_str("NUMBER(MODULES_VADDR)=0x%lx\n", MODULES_VADDR); > + vmcoreinfo_append_str("NUMBER(MODULES_END)=0x%lx\n", MODULES_END); > + vmcoreinfo_append_str("NUMBER(VMALLOC_START)=0x%lx\n", VMALLOC_START); > + vmcoreinfo_append_str("NUMBER(VMALLOC_END)=0x%lx\n", VMALLOC_END); > + vmcoreinfo_append_str("NUMBER(VMEMMAP_START)=0x%lx\n", VMEMMAP_START); > + vmcoreinfo_append_str("NUMBER(VMEMMAP_END)=0x%lx\n", VMEMMAP_END); > vmcoreinfo_append_str("NUMBER(kimage_voffset)=0x%llx\n", > kimage_voffset); > vmcoreinfo_append_str("NUMBER(PHYS_OFFSET)=0x%llx\n", > -- > 2.30.2 >
On Wed, Feb 09, 2022 at 09:26:42AM +0000, Huang Shijie wrote: > The following interrelated ranges are needed by the kdump crash tool: > MODULES_VADDR ~ MODULES_END, > VMALLOC_START ~ VMALLOC_END, > VMEMMAP_START ~ VMEMMAP_END > > Since these values change from time to time, it is preferable to export > them via vmcoreinfo than to change the crash's code frequently. Please can you explain _why_ they are needed? Will
Hi Will, On Tue, Feb 15, 2022 at 04:44:23PM +0000, Will Deacon wrote: > On Wed, Feb 09, 2022 at 09:26:42AM +0000, Huang Shijie wrote: > > The following interrelated ranges are needed by the kdump crash tool: > > MODULES_VADDR ~ MODULES_END, > > VMALLOC_START ~ VMALLOC_END, > > VMEMMAP_START ~ VMEMMAP_END > > > > Since these values change from time to time, it is preferable to export > > them via vmcoreinfo than to change the crash's code frequently. > > Please can you explain _why_ they are needed? The current Crash code is still based at kernel v4.9. The virtual memory layout looks like this: +--------------------------------------------------------------------+ | KASAN | MODULE | VMALLOC | .... | VMEMMAP | +--------------------------------------------------------------------+ The Crash uses MODULES range to set the VMALLOC ranges. If the ranges are wrong, Crash will _NOT_ works well for some latest kernel ,such as v5.11 later. (Please correct me if I am wrong). It seems the VMEMMAP range is less important. I list all the changes since v4.9 to now: 1.) The current crash code is based at kernel v4.9. The virtual memory layout looks like this: +--------------------------------------------------------------------+ | KASAN | MODULE | VMALLOC | .... | VMEMMAP | +--------------------------------------------------------------------+ The macros are: #define MODULES_VADDR (VA_START + KASAN_SHADOW_SIZE) #define MODULES_END (MODULES_VADDR + MODULES_VSIZE) #define VMALLOC_START (MODULES_END) #define VMALLOC_END (PAGE_OFFSET - PUD_SIZE - VMEMMAP_SIZE - SZ_64K) #define VMEMMAP_START (PAGE_OFFSET - VMEMMAP_SIZE) 2.) In the kernel v5.0, the patch will add a new BFP JIT region: "91fc957c9b1d arm64/bpf: don't allocate BPF JIT programs in module memory" The virtual memory layout looks like this: +--------------------------------------------------------------------+ | KASAN | BPF_JIT | MODULE | VMALLOC | .... | VMEMMAP | +--------------------------------------------------------------------+ The macros are: #define MODULES_VADDR (BPF_JIT_REGION_END) #define MODULES_END (MODULES_VADDR + MODULES_VSIZE) #define VMALLOC_START (MODULES_END) #define VMALLOC_END (PAGE_OFFSET - PUD_SIZE - VMEMMAP_SIZE - SZ_64K) #define VMEMMAP_START (PAGE_OFFSET - VMEMMAP_SIZE) The layout does not changed until v5.4. 3.) In the kernel v5.4, several patches changes the layout, such as: "ce3aaed87344 arm64: mm: Modify calculation of VMEMMAP_SIZE" "14c127c957c1 arm64: mm: Flip kernel VA space" and the virtual memory layout looks like this: +--------------------------------------------------------------------+ | KASAN | BPF_JIT | MODULE | VMALLOC | .... | VMEMMAP | +--------------------------------------------------------------------+ The macros are: #define MODULES_VADDR (BPF_JIT_REGION_END) #define MODULES_END (MODULES_VADDR + MODULES_VSIZE) #define VMALLOC_START (MODULES_END) #define VMALLOC_END (- PUD_SIZE - VMEMMAP_SIZE - SZ_64K) #define VMEMMAP_START (-VMEMMAP_SIZE - SZ_2M) 4.) In the kernel v5.11, several patches changes the layout, such as: "9ad7c6d5e75b arm64: mm: tidy up top of kernel VA space" "f4693c2716b3 arm64: mm: extend linear region for 52-bit VA configurations" and the virtual memory layout looks like this: +--------------------------------------------------------------------+ | BPF_JIT | MODULE | VMALLOC | .... | VMEMMAP | +--------------------------------------------------------------------+ The macros are: #define MODULES_VADDR (BPF_JIT_REGION_END) #define MODULES_END (MODULES_VADDR + MODULES_VSIZE) #define VMALLOC_START (MODULES_END) #define VMALLOC_END (VMEMMAP_START - SZ_256M) #define VMEMMAP_START (-(UL(1) << (VA_BITS - VMEMMAP_SHIFT))) #define VMEMMAP_END (VMEMMAP_START + VMEMMAP_SIZE) 5.) In the kernel v5.16, after the patch "b89ddf4cca43 arm64/bpf: Remove 128MB limit for BPF JIT programs" the virtual memory layout looks like this: +--------------------------------------------------------------------+ | MODULE | VMALLOC | .... | VMEMMAP | +--------------------------------------------------------------------+ The macros are: #define MODULES_VADDR (_PAGE_END(VA_BITS_MIN)) #define MODULES_END (MODULES_VADDR + MODULES_VSIZE) #define VMALLOC_START (MODULES_END) #define VMALLOC_END (VMEMMAP_START - SZ_256M) #define VMEMMAP_START (-(UL(1) << (VA_BITS - VMEMMAP_SHIFT))) #define VMEMMAP_END (VMEMMAP_START + VMEMMAP_SIZE) BTW:I am currently coding a patch for the Crash to update all the ranges to the latest kernel version v5.17-rc4. Thanks Huang Shijie
On Wed, Feb 16, 2022 at 09:28:49AM +0000, Huang Shijie wrote: > Hi Will, > On Tue, Feb 15, 2022 at 04:44:23PM +0000, Will Deacon wrote: > > On Wed, Feb 09, 2022 at 09:26:42AM +0000, Huang Shijie wrote: > > > The following interrelated ranges are needed by the kdump crash tool: > > > MODULES_VADDR ~ MODULES_END, > > > VMALLOC_START ~ VMALLOC_END, > > > VMEMMAP_START ~ VMEMMAP_END > > > > > > Since these values change from time to time, it is preferable to export > > > them via vmcoreinfo than to change the crash's code frequently. > > > > Please can you explain _why_ they are needed? > > The current Crash code is still based at kernel v4.9. > The virtual memory layout looks like this: > +--------------------------------------------------------------------+ > | KASAN | MODULE | VMALLOC | .... | VMEMMAP | > +--------------------------------------------------------------------+ > > The Crash uses MODULES range to set the VMALLOC ranges. > If the ranges are wrong, Crash will _NOT_ works well for some latest kernel > ,such as v5.11 later. (Please correct me if I am wrong). > It seems the VMEMMAP range is less important. [...] > 5.) In the kernel v5.16, after the patch > "b89ddf4cca43 arm64/bpf: Remove 128MB limit for BPF JIT programs" > the virtual memory layout looks like this: > > +--------------------------------------------------------------------+ > | MODULE | VMALLOC | .... | VMEMMAP | > +--------------------------------------------------------------------+ > > The macros are: > #define MODULES_VADDR (_PAGE_END(VA_BITS_MIN)) > #define MODULES_END (MODULES_VADDR + MODULES_VSIZE) > > #define VMALLOC_START (MODULES_END) > #define VMALLOC_END (VMEMMAP_START - SZ_256M) > > #define VMEMMAP_START (-(UL(1) << (VA_BITS - VMEMMAP_SHIFT))) > #define VMEMMAP_END (VMEMMAP_START + VMEMMAP_SIZE) > > > BTW:I am currently coding a patch for the Crash to update all the ranges to > the latest kernel version v5.17-rc4. Thanks for digging up all of the kernel memory map changes and taking the time to explain the macros. However, all I'm really after is something in the commit message of the patch which explains what is broken without this patch. What does crash use this information for, and what doesn't work at the moment? Cheers, Will
Hi Will, CC Kazu and Lianbo. On Wed, Feb 16, 2022 at 12:40:27PM +0000, Will Deacon wrote: > On Wed, Feb 16, 2022 at 09:28:49AM +0000, Huang Shijie wrote: > > Hi Will, > > On Tue, Feb 15, 2022 at 04:44:23PM +0000, Will Deacon wrote: > > > On Wed, Feb 09, 2022 at 09:26:42AM +0000, Huang Shijie wrote: > > > > The following interrelated ranges are needed by the kdump crash tool: > > > > MODULES_VADDR ~ MODULES_END, > > > > VMALLOC_START ~ VMALLOC_END, > > > > VMEMMAP_START ~ VMEMMAP_END > > > > > > > > Since these values change from time to time, it is preferable to export > > > > them via vmcoreinfo than to change the crash's code frequently. > > > > > > Please can you explain _why_ they are needed? > > > > The current Crash code is still based at kernel v4.9. > > The virtual memory layout looks like this: > > +--------------------------------------------------------------------+ > > | KASAN | MODULE | VMALLOC | .... | VMEMMAP | > > +--------------------------------------------------------------------+ > > > > The Crash uses MODULES range to set the VMALLOC ranges. > > If the ranges are wrong, Crash will _NOT_ works well for some latest kernel > > ,such as v5.11 later. (Please correct me if I am wrong). > > It seems the VMEMMAP range is less important. > > [...] > > > 5.) In the kernel v5.16, after the patch > > "b89ddf4cca43 arm64/bpf: Remove 128MB limit for BPF JIT programs" > > the virtual memory layout looks like this: > > > > +--------------------------------------------------------------------+ > > | MODULE | VMALLOC | .... | VMEMMAP | > > +--------------------------------------------------------------------+ > > > > The macros are: > > #define MODULES_VADDR (_PAGE_END(VA_BITS_MIN)) > > #define MODULES_END (MODULES_VADDR + MODULES_VSIZE) > > > > #define VMALLOC_START (MODULES_END) > > #define VMALLOC_END (VMEMMAP_START - SZ_256M) > > > > #define VMEMMAP_START (-(UL(1) << (VA_BITS - VMEMMAP_SHIFT))) > > #define VMEMMAP_END (VMEMMAP_START + VMEMMAP_SIZE) > > > > > > BTW:I am currently coding a patch for the Crash to update all the ranges to > > the latest kernel version v5.17-rc4. > > Thanks for digging up all of the kernel memory map changes and taking the > time to explain the macros. However, all I'm really after is something in > the commit message of the patch which explains what is broken without this This kernel patch does not break anything. It just makes the Crash easy to maintain. > patch. What does crash use this information for, and what doesn't work at > the moment? I know two cases now: 1.) The Crash uses MODULES/VMALLOC/VMEMMAP ranges in arm64_IS_VMALLOC_ADDR(): https://github.com/crash-utility/crash/blob/master/arm64.c#L4104 If arm64_IS_VMALLOC_ADDR() does not work correctly, the internal code may get wrong results. 2.) The "help -m" gets wrong output about MODULES/VMALLOC/VMEMMAP ranges. The guy who use the Use the Crash to debug a vmcore, will get the wrong information of the kernel panic. Thanks Huang Shijie
On Thu, Feb 17, 2022 at 10:02:03AM +0000, Huang Shijie wrote: > > Thanks for digging up all of the kernel memory map changes and taking the > > time to explain the macros. However, all I'm really after is something in > > the commit message of the patch which explains what is broken without this > This kernel patch does not break anything. I mean this kernel patch is not a bugfix, it is just make the ARM64 ecosystem (including the Crash) a little better.. The Crash can just parse out the MODULES/VMALLOC/VMALLOC ranges from the vmcore file, no need to calculate them in a very _hard_ way. Thanks Huang Shijie
On Wed, 9 Feb 2022 09:26:42 +0000, Huang Shijie wrote: > The following interrelated ranges are needed by the kdump crash tool: > MODULES_VADDR ~ MODULES_END, > VMALLOC_START ~ VMALLOC_END, > VMEMMAP_START ~ VMEMMAP_END > > Since these values change from time to time, it is preferable to export > them via vmcoreinfo than to change the crash's code frequently. > > [...] Applied to arm64 (for-next/mm), thanks! [1/1] arm64: crash_core: Export MODULES, VMALLOC, and VMEMMAP ranges https://git.kernel.org/arm64/c/2369f171d5c5 Cheers,
diff --git a/Documentation/admin-guide/kdump/vmcoreinfo.rst b/Documentation/admin-guide/kdump/vmcoreinfo.rst index 3861a25faae1..a339af45a22e 100644 --- a/Documentation/admin-guide/kdump/vmcoreinfo.rst +++ b/Documentation/admin-guide/kdump/vmcoreinfo.rst @@ -494,6 +494,14 @@ architecture which is used to lookup the page-tables for the Virtual addresses in the higher VA range (refer to ARMv8 ARM document for more details). +MODULES_VADDR|MODULES_END|VMALLOC_START|VMALLOC_END|VMEMMAP_START|VMEMMAP_END +------------- + +Used to get the correct ranges: + MODULES_VADDR ~ MODULES_END-1 : Kernel module space. + VMALLOC_START ~ VMALLOC_END-1 : vmalloc() / ioremap() space. + VMEMMAP_START ~ VMEMMAP_END-1 : vmemmap region, used for struct page array. + arm === diff --git a/arch/arm64/kernel/crash_core.c b/arch/arm64/kernel/crash_core.c index 314391a156ee..2b65aae332ce 100644 --- a/arch/arm64/kernel/crash_core.c +++ b/arch/arm64/kernel/crash_core.c @@ -20,6 +20,12 @@ void arch_crash_save_vmcoreinfo(void) { VMCOREINFO_NUMBER(VA_BITS); /* Please note VMCOREINFO_NUMBER() uses "%d", not "%x" */ + vmcoreinfo_append_str("NUMBER(MODULES_VADDR)=0x%lx\n", MODULES_VADDR); + vmcoreinfo_append_str("NUMBER(MODULES_END)=0x%lx\n", MODULES_END); + vmcoreinfo_append_str("NUMBER(VMALLOC_START)=0x%lx\n", VMALLOC_START); + vmcoreinfo_append_str("NUMBER(VMALLOC_END)=0x%lx\n", VMALLOC_END); + vmcoreinfo_append_str("NUMBER(VMEMMAP_START)=0x%lx\n", VMEMMAP_START); + vmcoreinfo_append_str("NUMBER(VMEMMAP_END)=0x%lx\n", VMEMMAP_END); vmcoreinfo_append_str("NUMBER(kimage_voffset)=0x%llx\n", kimage_voffset); vmcoreinfo_append_str("NUMBER(PHYS_OFFSET)=0x%llx\n",
The following interrelated ranges are needed by the kdump crash tool: MODULES_VADDR ~ MODULES_END, VMALLOC_START ~ VMALLOC_END, VMEMMAP_START ~ VMEMMAP_END Since these values change from time to time, it is preferable to export them via vmcoreinfo than to change the crash's code frequently. Signed-off-by: Huang Shijie <shijie@os.amperecomputing.com> --- Documentation/admin-guide/kdump/vmcoreinfo.rst | 8 ++++++++ arch/arm64/kernel/crash_core.c | 6 ++++++ 2 files changed, 14 insertions(+)