| Message ID | 20220414115720.1887-1-thunder.leizhen@huawei.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | support reserving crashkernel above 4G on arm64 kdump | expand |
For the series: Tested-by: Dave Kleikamp <dave.kleikamp@oracle.com> Thanks for keeping this patch set alive. On 4/14/22 6:57AM, Zhen Lei wrote: > Changes since [v21]: > 1. Update the commit message of patch 1 and 5. > 2. Add some comments for reserve_crashkernel() in patch 5. > > Thanks to Baoquan He and John Donnelly for their review comments. > > Because v5.18-rc1 has added a new patch > commit 031495635b46 ("arm64: Do not defer reserve_crashkernel() for platforms with no DMA memory zones") > There are many new scenarios: > 1) The mappings may be block or page-level. > 2) The call to reserve_crashkernel() may or may not be deferred. > 3) The the upper limit of DMA address may be 4G, or less than 4G. Or the > upper limit of physical memory, because SMMU can do the mapping. > > The code of patch 1-2, 8-9 keep no change, because the above-mentioned issues are not involved. > The code of patch 5 only makes the following changes: > - if (crash_base >= SZ_4G) > + /* > + * When both CONFIG_ZONE_DMA and CONFIG_ZONE_DMA32 are disabled, the > + * CRASH_ADDR_LOW_MAX equals the upper limit of physical memory, so > + * the 'crash_base' of high memory can not exceed it. To follow the > + * description of "crashkernel=X,high" option, add below 'high' > + * condition to make sure the crash low memory will be reserved. > + */ > + if ((crash_base >= CRASH_ADDR_LOW_MAX) || high) { > Change SZ_4G to CRASH_ADDR_LOW_MAX, because arm64_dma_phys_limit may be less than > 4G or greater than 4G. The check 'high' is used for "crashkernel=X,high" and > "(crash_base >= CRASH_ADDR_LOW_MAX)" is used for "crashkernel=X[@offset]". > > Patch 3-4 to allow block mappings for memory above 4G. > Patch 6-7 to support only crash high memory or fixed memory range specified by > crashkernel=X@offset use page-level mapping, to allow other areas use block mapping. > These four patches are for performance optimization purposes. For details about the > technical feasibility analysis, please see the commit messages. > > Now the implementation of arm64 is very different from that of x86. It's no longer > suitable for both of them to share code. > > > > Changes since [v20]: > 1. Check whether crashkernel=Y,low is incorrectly configured or not configured. Do different processing. > 2. Share the existing description of x86. The configuration of arm64 is the same as that of x86. > 3. Define the value of macro CRASH_ADDR_HIGH_MAX as memblock.current_limit, instead of MEMBLOCK_ALLOC_ACCESSIBLE. > 4. To improve readability, some lightweight code adjustments have been made to reserve_craskernel(), including comments. > 5. The defined value of DEFAULT_CRASH_KERNEL_LOW_SIZE reconsiders swiotlb, just like x86, to share documents. > > Thanks to Baoquan He for his careful review. > > The test cases are as follows: (Please update the kexec tool to the latest version) > 1) crashkernel=4G //high=4G, low=256M > 2) crashkernel=4G crashkernel=512M,high crashkernel=512M,low //high=4G, low=256M, high and low are ignored > 3) crashkernel=4G crashkernel=512M,high //high=4G, low=256M, high is ignored > 4) crashkernel=4G crashkernel=512M,low //high=4G, low=256M, low is ignored > 5) crashkernel=4G@0xe0000000 //high=0G, low=0M, cannot allocate, failed > 6) crashkernel=512M //high=0G, low=512M > 7) crashkernel=128M //high=0G, low=128M > 8) crashkernel=512M@0xde000000 //512M@3552M //high=0G, low=512M > 9) crashkernel=4G,high //high=4G, low=256M > a) crashkernel=4G,high crashkernel=512M,low //high=4G, low=512M > b) crashkernel=512M,high crashkernel=128M,low //high=512M, low=128M > c) crashkernel=128M,high //high=128M, low=256M > d) crashkernel=512M,low //high=0G, low=0M, invalid > e) crashkernel=512M,high crashkernel=0,low //high=512M, low=0M > f) crashkernel=4G,high crashkernel=ab,low //high=0G, low=0M, invalid > > > Changes since [v19]: > 1. Temporarily stop making reserve_crashkernel[_low]() generic. There are a > lot of details need to be considered, which can take a long time. Because > "make generic" does not add new functions and does not improve performance, > maybe I should say it's just a cleanup. So by stripping it out and leaving > it for other patches later, we can aggregate the changes to the main functions. > 2. Use insert_resource() to replace request_resource(), this not only simplifies > the code, but also reduces the differences between arm64 and x86 implementations. > 3. As commit 157752d84f5d ("kexec: use Crash kernel for Crash kernel low") do for > x86, we can also extend kexec-tools for arm64, and it's currently applied. See: > https://www.spinics.net/lists/kexec/msg28284.html > > Thank you very much, Borislav Petkov, for so many valuable comments. > > Changes since [v17]: v17 --> v19 > 1. Patch 0001-0004 > Introduce generic parse_crashkernel_high_low() to bring the parsing of > "crashkernel=X,high" and the parsing of "crashkernel=X,low" together, > then use it instead of the call to parse_crashkernel_{high|low}(). Two > confusing parameters of parse_crashkernel_{high|low}() are deleted. > > I previously sent these four patches separately: > [1] https://lkml.org/lkml/2021/12/25/40 > 2. Patch 0005-0009 > Introduce generic reserve_crashkernel_mem[_low](), the implementation of > these two functions is based on function reserve_crashkernel[_low]() in > arch/x86/kernel/setup.c. There is no functional change for x86. > 1) The check position of xen_pv_domain() does not change. > 2) Still 1M alignment for crash kernel fixed region, when 'base' is specified. > > To avoid compilation problems on other architectures: patch 0004 moves > the definition of global variable crashk[_low]_res from kexec_core.c to > crash_core.c, and provide default definitions for all macros involved, a > particular platform can redefine these macros to override the default > values. > 3. 0010, only one line of comment was changed. > 4. 0011 > 1) crashk_low_res may also a valid reserved memory, should be checked > in crash_is_nosave(), see arch/arm64/kernel/machine_kexec. > 2) Drop memblock_mark_nomap() for crashk_low_res, because of: > 2687275a5843 arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required > 3) Also call kmemleak_ignore_phys() for crashk_low_res, because of: > 85f58eb18898 arm64: kdump: Skip kmemleak scan reserved memory for kdump > 5. 0012, slightly rebased, because the following patch is applied in advance. > https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git/commit/?h=dt/linus&id=8347b41748c3019157312fbe7f8a6792ae396eb7 > 6. 0013, no change. > > Others: > 1. Discard add ARCH_WANT_RESERVE_CRASH_KERNEL > 2. When allocating crash low memory, the start address still starts from 0. > low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0, CRASH_ADDR_LOW_MAX); > 3. Discard change (1ULL << 32) to CRASH_ADDR_LOW_MAX. > 4. Ensure the check position of xen_pv_domain() have no change. > 5. Except patch 0010 and 0012, all "Tested-by", "Reviewed-by", "Acked-by" are removed. > 6. Update description. > > > > Changes since [v16] > - Because no functional changes in this version, so add > "Tested-by: Dave Kleikamp <dave.kleikamp@oracle.com>" for patch 1-9 > - Add "Reviewed-by: Rob Herring <robh@kernel.org>" for patch 8 > - Update patch 9 based on the review comments of Rob Herring > - As Catalin Marinas's suggestion, merge the implementation of > ARCH_WANT_RESERVE_CRASH_KERNEL into patch 5. Ensure that the > contents of X86 and ARM64 do not overlap, and reduce unnecessary > temporary differences. > > Changes since [v15] > - Aggregate the processing of "linux,usable-memory-range" into one function. > Only patch 9-10 have been updated. > > Changes since [v14] > - Recovering the requirement that the CrashKernel memory regions on X86 > only requires 1 MiB alignment. > - Combine patches 5 and 6 in v14 into one. The compilation warning fixed > by patch 6 was introduced by patch 5 in v14. > - As with crashk_res, crashk_low_res is also processed by > crash_exclude_mem_range() in patch 7. > - Due to commit b261dba2fdb2 ("arm64: kdump: Remove custom linux,usable-memory-range handling") > has removed the architecture-specific code, extend the property "linux,usable-memory-range" > in the platform-agnostic FDT core code. See patch 9. > - Discard the x86 description update in the document, because the description > has been updated by commit b1f4c363666c ("Documentation: kdump: update kdump guide"). > - Change "arm64" to "ARM64" in Doc. > > > Changes since [v13] > - Rebased on top of 5.11-rc5. > - Introduce config CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL. > Since reserve_crashkernel[_low]() implementations are quite similar on > other architectures, so have CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL in > arch/Kconfig and select this by X86 and ARM64. > - Some minor cleanup. > > Changes since [v12] > - Rebased on top of 5.10-rc1. > - Keep CRASH_ALIGN as 16M suggested by Dave. > - Drop patch "kdump: add threshold for the required memory". > - Add Tested-by from John. > > Changes since [v11] > - Rebased on top of 5.9-rc4. > - Make the function reserve_crashkernel() of x86 generic. > Suggested by Catalin, make the function reserve_crashkernel() of x86 generic > and arm64 use the generic version to reimplement crashkernel=X. > > Changes since [v10] > - Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin. > > Changes since [v9] > - Patch 1 add Acked-by from Dave. > - Update patch 5 according to Dave's comments. > - Update chosen schema. > > Changes since [v8] > - Reuse DT property "linux,usable-memory-range". > Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the low > memory region. > - Fix kdump broken with ZONE_DMA reintroduced. > - Update chosen schema. > > Changes since [v7] > - Move x86 CRASH_ALIGN to 2M > Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. > - Update Documentation/devicetree/bindings/chosen.txt. > Add corresponding documentation to Documentation/devicetree/bindings/chosen.txt > suggested by Arnd. > - Add Tested-by from Jhon and pk. > > Changes since [v6] > - Fix build errors reported by kbuild test robot. > > Changes since [v5] > - Move reserve_crashkernel_low() into kernel/crash_core.c. > - Delete crashkernel=X,high. > - Modify crashkernel=X,low. > If crashkernel=X,low is specified simultaneously, reserve spcified size low > memory for crash kdump kernel devices firstly and then reserve memory above 4G. > In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then > pass to crash dump kernel by DT property "linux,low-memory-range". > - Update Documentation/admin-guide/kdump/kdump.rst. > > Changes since [v4] > - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. > > Changes since [v3] > - Add memblock_cap_memory_ranges back for multiple ranges. > - Fix some compiling warnings. > > Changes since [v2] > - Split patch "arm64: kdump: support reserving crashkernel above 4G" as > two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate > patch. > > Changes since [v1]: > - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. > - Remove memblock_cap_memory_ranges() i added in v1 and implement that > in fdt_enforce_memory_region(). > There are at most two crash kernel regions, for two crash kernel regions > case, we cap the memory range [min(regs[*].start), max(regs[*].end)] > and then remove the memory range in the middle. > > v1: > There are following issues in arm64 kdump: > 1. We use crashkernel=X to reserve crashkernel below 4G, which > will fail when there is no enough low memory. > 2. If reserving crashkernel above 4G, in this case, crash dump > kernel will boot failure because there is no low memory available > for allocation. > > To solve these issues, change the behavior of crashkernel=X. > crashkernel=X tries low allocation in DMA zone and fall back to high > allocation if it fails. > > We can also use "crashkernel=X,high" to select a high region above > DMA zone, which also tries to allocate at least 256M low memory in > DMA zone automatically and "crashkernel=Y,low" can be used to allocate > specified size low memory. > > When reserving crashkernel in high memory, some low memory is reserved > for crash dump kernel devices. So there may be two regions reserved for > crash dump kernel. > In order to distinct from the high region and make no effect to the use > of existing kexec-tools, rename the low region as "Crash kernel (low)", > and pass the low region by reusing DT property > "linux,usable-memory-range". We made the low memory region as the last > range of "linux,usable-memory-range" to keep compatibility with existing > user-space and older kdump kernels. > > Besides, we need to modify kexec-tools: > arm64: support more than one crash kernel regions(see [1]) > > Another update is document about DT property 'linux,usable-memory-range': > schemas: update 'linux,usable-memory-range' node schema(see [2]) > > > [1]: https://www.spinics.net/lists/kexec/msg28226.html > [2]: https://github.com/robherring/dt-schema/pull/19 > [v1]: https://lkml.org/lkml/2019/4/2/1174 > [v2]: https://lkml.org/lkml/2019/4/9/86 > [v3]: https://lkml.org/lkml/2019/4/9/306 > [v4]: https://lkml.org/lkml/2019/4/15/273 > [v5]: https://lkml.org/lkml/2019/5/6/1360 > [v6]: https://lkml.org/lkml/2019/8/30/142 > [v7]: https://lkml.org/lkml/2019/12/23/411 > [v8]: https://lkml.org/lkml/2020/5/21/213 > [v9]: https://lkml.org/lkml/2020/6/28/73 > [v10]: https://lkml.org/lkml/2020/7/2/1443 > [v11]: https://lkml.org/lkml/2020/8/1/150 > [v12]: https://lkml.org/lkml/2020/9/7/1037 > [v13]: https://lkml.org/lkml/2020/10/31/34 > [v14]: https://lkml.org/lkml/2021/1/30/53 > [v15]: https://lkml.org/lkml/2021/10/19/1405 > [v16]: https://lkml.org/lkml/2021/11/23/435 > [v17]: https://lkml.org/lkml/2021/12/10/38 > [v18]: https://lkml.org/lkml/2021/12/22/424 > [v19]: https://lkml.org/lkml/2021/12/28/203 > [v20]: https://lkml.org/lkml/2022/1/24/167 > [v21]: https://lkml.org/lkml/2022/2/26/350 > > Chen Zhou (2): > arm64: kdump: Reimplement crashkernel=X > of: fdt: Add memory for devices by DT property > "linux,usable-memory-range" > > Zhen Lei (7): > kdump: return -ENOENT if required cmdline option does not exist > arm64: Use insert_resource() to simplify code > arm64: kdump: Remove some redundant checks in map_mem() > arm64: kdump: Don't force page-level mappings for memory above 4G > arm64: kdump: Use page-level mapping for the high memory of > crashkernel > arm64: kdump: Try not to use NO_BLOCK_MAPPINGS for memory under 4G > docs: kdump: Update the crashkernel description for arm64 > > .../admin-guide/kernel-parameters.txt | 8 +- > arch/arm64/include/asm/kexec.h | 2 + > arch/arm64/kernel/machine_kexec.c | 9 +- > arch/arm64/kernel/machine_kexec_file.c | 12 +- > arch/arm64/kernel/setup.c | 17 +- > arch/arm64/mm/init.c | 204 +++++++++++++++++- > arch/arm64/mm/mmu.c | 68 +++--- > drivers/of/fdt.c | 33 ++- > kernel/crash_core.c | 3 +- > 9 files changed, 278 insertions(+), 78 deletions(-) >
Hi, Catalin Marinas and Will Deacon: Do you have time to review patches 3-4 and 6-7? These patches are a little more relevant to the arm64 architecture. Hi, Baoquan He: Can you review patches 1 and 5 again? On 2022/4/14 19:57, Zhen Lei wrote: > Changes since [v21]: > 1. Update the commit message of patch 1 and 5. > 2. Add some comments for reserve_crashkernel() in patch 5. > > Thanks to Baoquan He and John Donnelly for their review comments. > > Because v5.18-rc1 has added a new patch > commit 031495635b46 ("arm64: Do not defer reserve_crashkernel() for platforms with no DMA memory zones") > There are many new scenarios: > 1) The mappings may be block or page-level. > 2) The call to reserve_crashkernel() may or may not be deferred. > 3) The the upper limit of DMA address may be 4G, or less than 4G. Or the > upper limit of physical memory, because SMMU can do the mapping. > > The code of patch 1-2, 8-9 keep no change, because the above-mentioned issues are not involved. > The code of patch 5 only makes the following changes: > - if (crash_base >= SZ_4G) > + /* > + * When both CONFIG_ZONE_DMA and CONFIG_ZONE_DMA32 are disabled, the > + * CRASH_ADDR_LOW_MAX equals the upper limit of physical memory, so > + * the 'crash_base' of high memory can not exceed it. To follow the > + * description of "crashkernel=X,high" option, add below 'high' > + * condition to make sure the crash low memory will be reserved. > + */ > + if ((crash_base >= CRASH_ADDR_LOW_MAX) || high) { > Change SZ_4G to CRASH_ADDR_LOW_MAX, because arm64_dma_phys_limit may be less than > 4G or greater than 4G. The check 'high' is used for "crashkernel=X,high" and > "(crash_base >= CRASH_ADDR_LOW_MAX)" is used for "crashkernel=X[@offset]". > > Patch 3-4 to allow block mappings for memory above 4G. > Patch 6-7 to support only crash high memory or fixed memory range specified by > crashkernel=X@offset use page-level mapping, to allow other areas use block mapping. > These four patches are for performance optimization purposes. For details about the > technical feasibility analysis, please see the commit messages. > > Now the implementation of arm64 is very different from that of x86. It's no longer > suitable for both of them to share code. > > > > Changes since [v20]: > 1. Check whether crashkernel=Y,low is incorrectly configured or not configured. Do different processing. > 2. Share the existing description of x86. The configuration of arm64 is the same as that of x86. > 3. Define the value of macro CRASH_ADDR_HIGH_MAX as memblock.current_limit, instead of MEMBLOCK_ALLOC_ACCESSIBLE. > 4. To improve readability, some lightweight code adjustments have been made to reserve_craskernel(), including comments. > 5. The defined value of DEFAULT_CRASH_KERNEL_LOW_SIZE reconsiders swiotlb, just like x86, to share documents. > > Thanks to Baoquan He for his careful review. > > The test cases are as follows: (Please update the kexec tool to the latest version) > 1) crashkernel=4G //high=4G, low=256M > 2) crashkernel=4G crashkernel=512M,high crashkernel=512M,low //high=4G, low=256M, high and low are ignored > 3) crashkernel=4G crashkernel=512M,high //high=4G, low=256M, high is ignored > 4) crashkernel=4G crashkernel=512M,low //high=4G, low=256M, low is ignored > 5) crashkernel=4G@0xe0000000 //high=0G, low=0M, cannot allocate, failed > 6) crashkernel=512M //high=0G, low=512M > 7) crashkernel=128M //high=0G, low=128M > 8) crashkernel=512M@0xde000000 //512M@3552M //high=0G, low=512M > 9) crashkernel=4G,high //high=4G, low=256M > a) crashkernel=4G,high crashkernel=512M,low //high=4G, low=512M > b) crashkernel=512M,high crashkernel=128M,low //high=512M, low=128M > c) crashkernel=128M,high //high=128M, low=256M > d) crashkernel=512M,low //high=0G, low=0M, invalid > e) crashkernel=512M,high crashkernel=0,low //high=512M, low=0M > f) crashkernel=4G,high crashkernel=ab,low //high=0G, low=0M, invalid > > > Changes since [v19]: > 1. Temporarily stop making reserve_crashkernel[_low]() generic. There are a > lot of details need to be considered, which can take a long time. Because > "make generic" does not add new functions and does not improve performance, > maybe I should say it's just a cleanup. So by stripping it out and leaving > it for other patches later, we can aggregate the changes to the main functions. > 2. Use insert_resource() to replace request_resource(), this not only simplifies > the code, but also reduces the differences between arm64 and x86 implementations. > 3. As commit 157752d84f5d ("kexec: use Crash kernel for Crash kernel low") do for > x86, we can also extend kexec-tools for arm64, and it's currently applied. See: > https://www.spinics.net/lists/kexec/msg28284.html > > Thank you very much, Borislav Petkov, for so many valuable comments. > > Changes since [v17]: v17 --> v19 > 1. Patch 0001-0004 > Introduce generic parse_crashkernel_high_low() to bring the parsing of > "crashkernel=X,high" and the parsing of "crashkernel=X,low" together, > then use it instead of the call to parse_crashkernel_{high|low}(). Two > confusing parameters of parse_crashkernel_{high|low}() are deleted. > > I previously sent these four patches separately: > [1] https://lkml.org/lkml/2021/12/25/40 > 2. Patch 0005-0009 > Introduce generic reserve_crashkernel_mem[_low](), the implementation of > these two functions is based on function reserve_crashkernel[_low]() in > arch/x86/kernel/setup.c. There is no functional change for x86. > 1) The check position of xen_pv_domain() does not change. > 2) Still 1M alignment for crash kernel fixed region, when 'base' is specified. > > To avoid compilation problems on other architectures: patch 0004 moves > the definition of global variable crashk[_low]_res from kexec_core.c to > crash_core.c, and provide default definitions for all macros involved, a > particular platform can redefine these macros to override the default > values. > 3. 0010, only one line of comment was changed. > 4. 0011 > 1) crashk_low_res may also a valid reserved memory, should be checked > in crash_is_nosave(), see arch/arm64/kernel/machine_kexec. > 2) Drop memblock_mark_nomap() for crashk_low_res, because of: > 2687275a5843 arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required > 3) Also call kmemleak_ignore_phys() for crashk_low_res, because of: > 85f58eb18898 arm64: kdump: Skip kmemleak scan reserved memory for kdump > 5. 0012, slightly rebased, because the following patch is applied in advance. > https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git/commit/?h=dt/linus&id=8347b41748c3019157312fbe7f8a6792ae396eb7 > 6. 0013, no change. > > Others: > 1. Discard add ARCH_WANT_RESERVE_CRASH_KERNEL > 2. When allocating crash low memory, the start address still starts from 0. > low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0, CRASH_ADDR_LOW_MAX); > 3. Discard change (1ULL << 32) to CRASH_ADDR_LOW_MAX. > 4. Ensure the check position of xen_pv_domain() have no change. > 5. Except patch 0010 and 0012, all "Tested-by", "Reviewed-by", "Acked-by" are removed. > 6. Update description. > > > > Changes since [v16] > - Because no functional changes in this version, so add > "Tested-by: Dave Kleikamp <dave.kleikamp@oracle.com>" for patch 1-9 > - Add "Reviewed-by: Rob Herring <robh@kernel.org>" for patch 8 > - Update patch 9 based on the review comments of Rob Herring > - As Catalin Marinas's suggestion, merge the implementation of > ARCH_WANT_RESERVE_CRASH_KERNEL into patch 5. Ensure that the > contents of X86 and ARM64 do not overlap, and reduce unnecessary > temporary differences. > > Changes since [v15] > - Aggregate the processing of "linux,usable-memory-range" into one function. > Only patch 9-10 have been updated. > > Changes since [v14] > - Recovering the requirement that the CrashKernel memory regions on X86 > only requires 1 MiB alignment. > - Combine patches 5 and 6 in v14 into one. The compilation warning fixed > by patch 6 was introduced by patch 5 in v14. > - As with crashk_res, crashk_low_res is also processed by > crash_exclude_mem_range() in patch 7. > - Due to commit b261dba2fdb2 ("arm64: kdump: Remove custom linux,usable-memory-range handling") > has removed the architecture-specific code, extend the property "linux,usable-memory-range" > in the platform-agnostic FDT core code. See patch 9. > - Discard the x86 description update in the document, because the description > has been updated by commit b1f4c363666c ("Documentation: kdump: update kdump guide"). > - Change "arm64" to "ARM64" in Doc. > > > Changes since [v13] > - Rebased on top of 5.11-rc5. > - Introduce config CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL. > Since reserve_crashkernel[_low]() implementations are quite similar on > other architectures, so have CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL in > arch/Kconfig and select this by X86 and ARM64. > - Some minor cleanup. > > Changes since [v12] > - Rebased on top of 5.10-rc1. > - Keep CRASH_ALIGN as 16M suggested by Dave. > - Drop patch "kdump: add threshold for the required memory". > - Add Tested-by from John. > > Changes since [v11] > - Rebased on top of 5.9-rc4. > - Make the function reserve_crashkernel() of x86 generic. > Suggested by Catalin, make the function reserve_crashkernel() of x86 generic > and arm64 use the generic version to reimplement crashkernel=X. > > Changes since [v10] > - Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin. > > Changes since [v9] > - Patch 1 add Acked-by from Dave. > - Update patch 5 according to Dave's comments. > - Update chosen schema. > > Changes since [v8] > - Reuse DT property "linux,usable-memory-range". > Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the low > memory region. > - Fix kdump broken with ZONE_DMA reintroduced. > - Update chosen schema. > > Changes since [v7] > - Move x86 CRASH_ALIGN to 2M > Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. > - Update Documentation/devicetree/bindings/chosen.txt. > Add corresponding documentation to Documentation/devicetree/bindings/chosen.txt > suggested by Arnd. > - Add Tested-by from Jhon and pk. > > Changes since [v6] > - Fix build errors reported by kbuild test robot. > > Changes since [v5] > - Move reserve_crashkernel_low() into kernel/crash_core.c. > - Delete crashkernel=X,high. > - Modify crashkernel=X,low. > If crashkernel=X,low is specified simultaneously, reserve spcified size low > memory for crash kdump kernel devices firstly and then reserve memory above 4G. > In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then > pass to crash dump kernel by DT property "linux,low-memory-range". > - Update Documentation/admin-guide/kdump/kdump.rst. > > Changes since [v4] > - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. > > Changes since [v3] > - Add memblock_cap_memory_ranges back for multiple ranges. > - Fix some compiling warnings. > > Changes since [v2] > - Split patch "arm64: kdump: support reserving crashkernel above 4G" as > two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate > patch. > > Changes since [v1]: > - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. > - Remove memblock_cap_memory_ranges() i added in v1 and implement that > in fdt_enforce_memory_region(). > There are at most two crash kernel regions, for two crash kernel regions > case, we cap the memory range [min(regs[*].start), max(regs[*].end)] > and then remove the memory range in the middle. > > v1: > There are following issues in arm64 kdump: > 1. We use crashkernel=X to reserve crashkernel below 4G, which > will fail when there is no enough low memory. > 2. If reserving crashkernel above 4G, in this case, crash dump > kernel will boot failure because there is no low memory available > for allocation. > > To solve these issues, change the behavior of crashkernel=X. > crashkernel=X tries low allocation in DMA zone and fall back to high > allocation if it fails. > > We can also use "crashkernel=X,high" to select a high region above > DMA zone, which also tries to allocate at least 256M low memory in > DMA zone automatically and "crashkernel=Y,low" can be used to allocate > specified size low memory. > > When reserving crashkernel in high memory, some low memory is reserved > for crash dump kernel devices. So there may be two regions reserved for > crash dump kernel. > In order to distinct from the high region and make no effect to the use > of existing kexec-tools, rename the low region as "Crash kernel (low)", > and pass the low region by reusing DT property > "linux,usable-memory-range". We made the low memory region as the last > range of "linux,usable-memory-range" to keep compatibility with existing > user-space and older kdump kernels. > > Besides, we need to modify kexec-tools: > arm64: support more than one crash kernel regions(see [1]) > > Another update is document about DT property 'linux,usable-memory-range': > schemas: update 'linux,usable-memory-range' node schema(see [2]) > > > [1]: https://www.spinics.net/lists/kexec/msg28226.html > [2]: https://github.com/robherring/dt-schema/pull/19 > [v1]: https://lkml.org/lkml/2019/4/2/1174 > [v2]: https://lkml.org/lkml/2019/4/9/86 > [v3]: https://lkml.org/lkml/2019/4/9/306 > [v4]: https://lkml.org/lkml/2019/4/15/273 > [v5]: https://lkml.org/lkml/2019/5/6/1360 > [v6]: https://lkml.org/lkml/2019/8/30/142 > [v7]: https://lkml.org/lkml/2019/12/23/411 > [v8]: https://lkml.org/lkml/2020/5/21/213 > [v9]: https://lkml.org/lkml/2020/6/28/73 > [v10]: https://lkml.org/lkml/2020/7/2/1443 > [v11]: https://lkml.org/lkml/2020/8/1/150 > [v12]: https://lkml.org/lkml/2020/9/7/1037 > [v13]: https://lkml.org/lkml/2020/10/31/34 > [v14]: https://lkml.org/lkml/2021/1/30/53 > [v15]: https://lkml.org/lkml/2021/10/19/1405 > [v16]: https://lkml.org/lkml/2021/11/23/435 > [v17]: https://lkml.org/lkml/2021/12/10/38 > [v18]: https://lkml.org/lkml/2021/12/22/424 > [v19]: https://lkml.org/lkml/2021/12/28/203 > [v20]: https://lkml.org/lkml/2022/1/24/167 > [v21]: https://lkml.org/lkml/2022/2/26/350 > > Chen Zhou (2): > arm64: kdump: Reimplement crashkernel=X > of: fdt: Add memory for devices by DT property > "linux,usable-memory-range" > > Zhen Lei (7): > kdump: return -ENOENT if required cmdline option does not exist > arm64: Use insert_resource() to simplify code > arm64: kdump: Remove some redundant checks in map_mem() > arm64: kdump: Don't force page-level mappings for memory above 4G > arm64: kdump: Use page-level mapping for the high memory of > crashkernel > arm64: kdump: Try not to use NO_BLOCK_MAPPINGS for memory under 4G > docs: kdump: Update the crashkernel description for arm64 > > .../admin-guide/kernel-parameters.txt | 8 +- > arch/arm64/include/asm/kexec.h | 2 + > arch/arm64/kernel/machine_kexec.c | 9 +- > arch/arm64/kernel/machine_kexec_file.c | 12 +- > arch/arm64/kernel/setup.c | 17 +- > arch/arm64/mm/init.c | 204 +++++++++++++++++- > arch/arm64/mm/mmu.c | 68 +++--- > drivers/of/fdt.c | 33 ++- > kernel/crash_core.c | 3 +- > 9 files changed, 278 insertions(+), 78 deletions(-) >
On 04/25/22 at 10:19am, Leizhen (ThunderTown) wrote: > > Hi, Catalin Marinas and Will Deacon: > Do you have time to review patches 3-4 and 6-7? These patches are a little more > relevant to the arm64 architecture. > > Hi, Baoquan He: > Can you review patches 1 and 5 again? Yes, and I will test and review the whole series if no one check the arm64 related changes. > > > On 2022/4/14 19:57, Zhen Lei wrote: > > Changes since [v21]: > > 1. Update the commit message of patch 1 and 5. > > 2. Add some comments for reserve_crashkernel() in patch 5. > > > > Thanks to Baoquan He and John Donnelly for their review comments. > > > > Because v5.18-rc1 has added a new patch > > commit 031495635b46 ("arm64: Do not defer reserve_crashkernel() for platforms with no DMA memory zones") > > There are many new scenarios: > > 1) The mappings may be block or page-level. > > 2) The call to reserve_crashkernel() may or may not be deferred. > > 3) The the upper limit of DMA address may be 4G, or less than 4G. Or the > > upper limit of physical memory, because SMMU can do the mapping. > > > > The code of patch 1-2, 8-9 keep no change, because the above-mentioned issues are not involved. > > The code of patch 5 only makes the following changes: > > - if (crash_base >= SZ_4G) > > + /* > > + * When both CONFIG_ZONE_DMA and CONFIG_ZONE_DMA32 are disabled, the > > + * CRASH_ADDR_LOW_MAX equals the upper limit of physical memory, so > > + * the 'crash_base' of high memory can not exceed it. To follow the > > + * description of "crashkernel=X,high" option, add below 'high' > > + * condition to make sure the crash low memory will be reserved. > > + */ > > + if ((crash_base >= CRASH_ADDR_LOW_MAX) || high) { > > Change SZ_4G to CRASH_ADDR_LOW_MAX, because arm64_dma_phys_limit may be less than > > 4G or greater than 4G. The check 'high' is used for "crashkernel=X,high" and > > "(crash_base >= CRASH_ADDR_LOW_MAX)" is used for "crashkernel=X[@offset]". > > > > Patch 3-4 to allow block mappings for memory above 4G. > > Patch 6-7 to support only crash high memory or fixed memory range specified by > > crashkernel=X@offset use page-level mapping, to allow other areas use block mapping. > > These four patches are for performance optimization purposes. For details about the > > technical feasibility analysis, please see the commit messages. > > > > Now the implementation of arm64 is very different from that of x86. It's no longer > > suitable for both of them to share code. > > > > > > > > Changes since [v20]: > > 1. Check whether crashkernel=Y,low is incorrectly configured or not configured. Do different processing. > > 2. Share the existing description of x86. The configuration of arm64 is the same as that of x86. > > 3. Define the value of macro CRASH_ADDR_HIGH_MAX as memblock.current_limit, instead of MEMBLOCK_ALLOC_ACCESSIBLE. > > 4. To improve readability, some lightweight code adjustments have been made to reserve_craskernel(), including comments. > > 5. The defined value of DEFAULT_CRASH_KERNEL_LOW_SIZE reconsiders swiotlb, just like x86, to share documents. > > > > Thanks to Baoquan He for his careful review. > > > > The test cases are as follows: (Please update the kexec tool to the latest version) > > 1) crashkernel=4G //high=4G, low=256M > > 2) crashkernel=4G crashkernel=512M,high crashkernel=512M,low //high=4G, low=256M, high and low are ignored > > 3) crashkernel=4G crashkernel=512M,high //high=4G, low=256M, high is ignored > > 4) crashkernel=4G crashkernel=512M,low //high=4G, low=256M, low is ignored > > 5) crashkernel=4G@0xe0000000 //high=0G, low=0M, cannot allocate, failed > > 6) crashkernel=512M //high=0G, low=512M > > 7) crashkernel=128M //high=0G, low=128M > > 8) crashkernel=512M@0xde000000 //512M@3552M //high=0G, low=512M > > 9) crashkernel=4G,high //high=4G, low=256M > > a) crashkernel=4G,high crashkernel=512M,low //high=4G, low=512M > > b) crashkernel=512M,high crashkernel=128M,low //high=512M, low=128M > > c) crashkernel=128M,high //high=128M, low=256M > > d) crashkernel=512M,low //high=0G, low=0M, invalid > > e) crashkernel=512M,high crashkernel=0,low //high=512M, low=0M > > f) crashkernel=4G,high crashkernel=ab,low //high=0G, low=0M, invalid > > > > > > Changes since [v19]: > > 1. Temporarily stop making reserve_crashkernel[_low]() generic. There are a > > lot of details need to be considered, which can take a long time. Because > > "make generic" does not add new functions and does not improve performance, > > maybe I should say it's just a cleanup. So by stripping it out and leaving > > it for other patches later, we can aggregate the changes to the main functions. > > 2. Use insert_resource() to replace request_resource(), this not only simplifies > > the code, but also reduces the differences between arm64 and x86 implementations. > > 3. As commit 157752d84f5d ("kexec: use Crash kernel for Crash kernel low") do for > > x86, we can also extend kexec-tools for arm64, and it's currently applied. See: > > https://www.spinics.net/lists/kexec/msg28284.html > > > > Thank you very much, Borislav Petkov, for so many valuable comments. > > > > Changes since [v17]: v17 --> v19 > > 1. Patch 0001-0004 > > Introduce generic parse_crashkernel_high_low() to bring the parsing of > > "crashkernel=X,high" and the parsing of "crashkernel=X,low" together, > > then use it instead of the call to parse_crashkernel_{high|low}(). Two > > confusing parameters of parse_crashkernel_{high|low}() are deleted. > > > > I previously sent these four patches separately: > > [1] https://lkml.org/lkml/2021/12/25/40 > > 2. Patch 0005-0009 > > Introduce generic reserve_crashkernel_mem[_low](), the implementation of > > these two functions is based on function reserve_crashkernel[_low]() in > > arch/x86/kernel/setup.c. There is no functional change for x86. > > 1) The check position of xen_pv_domain() does not change. > > 2) Still 1M alignment for crash kernel fixed region, when 'base' is specified. > > > > To avoid compilation problems on other architectures: patch 0004 moves > > the definition of global variable crashk[_low]_res from kexec_core.c to > > crash_core.c, and provide default definitions for all macros involved, a > > particular platform can redefine these macros to override the default > > values. > > 3. 0010, only one line of comment was changed. > > 4. 0011 > > 1) crashk_low_res may also a valid reserved memory, should be checked > > in crash_is_nosave(), see arch/arm64/kernel/machine_kexec. > > 2) Drop memblock_mark_nomap() for crashk_low_res, because of: > > 2687275a5843 arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required > > 3) Also call kmemleak_ignore_phys() for crashk_low_res, because of: > > 85f58eb18898 arm64: kdump: Skip kmemleak scan reserved memory for kdump > > 5. 0012, slightly rebased, because the following patch is applied in advance. > > https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git/commit/?h=dt/linus&id=8347b41748c3019157312fbe7f8a6792ae396eb7 > > 6. 0013, no change. > > > > Others: > > 1. Discard add ARCH_WANT_RESERVE_CRASH_KERNEL > > 2. When allocating crash low memory, the start address still starts from 0. > > low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0, CRASH_ADDR_LOW_MAX); > > 3. Discard change (1ULL << 32) to CRASH_ADDR_LOW_MAX. > > 4. Ensure the check position of xen_pv_domain() have no change. > > 5. Except patch 0010 and 0012, all "Tested-by", "Reviewed-by", "Acked-by" are removed. > > 6. Update description. > > > > > > > > Changes since [v16] > > - Because no functional changes in this version, so add > > "Tested-by: Dave Kleikamp <dave.kleikamp@oracle.com>" for patch 1-9 > > - Add "Reviewed-by: Rob Herring <robh@kernel.org>" for patch 8 > > - Update patch 9 based on the review comments of Rob Herring > > - As Catalin Marinas's suggestion, merge the implementation of > > ARCH_WANT_RESERVE_CRASH_KERNEL into patch 5. Ensure that the > > contents of X86 and ARM64 do not overlap, and reduce unnecessary > > temporary differences. > > > > Changes since [v15] > > - Aggregate the processing of "linux,usable-memory-range" into one function. > > Only patch 9-10 have been updated. > > > > Changes since [v14] > > - Recovering the requirement that the CrashKernel memory regions on X86 > > only requires 1 MiB alignment. > > - Combine patches 5 and 6 in v14 into one. The compilation warning fixed > > by patch 6 was introduced by patch 5 in v14. > > - As with crashk_res, crashk_low_res is also processed by > > crash_exclude_mem_range() in patch 7. > > - Due to commit b261dba2fdb2 ("arm64: kdump: Remove custom linux,usable-memory-range handling") > > has removed the architecture-specific code, extend the property "linux,usable-memory-range" > > in the platform-agnostic FDT core code. See patch 9. > > - Discard the x86 description update in the document, because the description > > has been updated by commit b1f4c363666c ("Documentation: kdump: update kdump guide"). > > - Change "arm64" to "ARM64" in Doc. > > > > > > Changes since [v13] > > - Rebased on top of 5.11-rc5. > > - Introduce config CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL. > > Since reserve_crashkernel[_low]() implementations are quite similar on > > other architectures, so have CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL in > > arch/Kconfig and select this by X86 and ARM64. > > - Some minor cleanup. > > > > Changes since [v12] > > - Rebased on top of 5.10-rc1. > > - Keep CRASH_ALIGN as 16M suggested by Dave. > > - Drop patch "kdump: add threshold for the required memory". > > - Add Tested-by from John. > > > > Changes since [v11] > > - Rebased on top of 5.9-rc4. > > - Make the function reserve_crashkernel() of x86 generic. > > Suggested by Catalin, make the function reserve_crashkernel() of x86 generic > > and arm64 use the generic version to reimplement crashkernel=X. > > > > Changes since [v10] > > - Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin. > > > > Changes since [v9] > > - Patch 1 add Acked-by from Dave. > > - Update patch 5 according to Dave's comments. > > - Update chosen schema. > > > > Changes since [v8] > > - Reuse DT property "linux,usable-memory-range". > > Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the low > > memory region. > > - Fix kdump broken with ZONE_DMA reintroduced. > > - Update chosen schema. > > > > Changes since [v7] > > - Move x86 CRASH_ALIGN to 2M > > Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. > > - Update Documentation/devicetree/bindings/chosen.txt. > > Add corresponding documentation to Documentation/devicetree/bindings/chosen.txt > > suggested by Arnd. > > - Add Tested-by from Jhon and pk. > > > > Changes since [v6] > > - Fix build errors reported by kbuild test robot. > > > > Changes since [v5] > > - Move reserve_crashkernel_low() into kernel/crash_core.c. > > - Delete crashkernel=X,high. > > - Modify crashkernel=X,low. > > If crashkernel=X,low is specified simultaneously, reserve spcified size low > > memory for crash kdump kernel devices firstly and then reserve memory above 4G. > > In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then > > pass to crash dump kernel by DT property "linux,low-memory-range". > > - Update Documentation/admin-guide/kdump/kdump.rst. > > > > Changes since [v4] > > - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. > > > > Changes since [v3] > > - Add memblock_cap_memory_ranges back for multiple ranges. > > - Fix some compiling warnings. > > > > Changes since [v2] > > - Split patch "arm64: kdump: support reserving crashkernel above 4G" as > > two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate > > patch. > > > > Changes since [v1]: > > - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. > > - Remove memblock_cap_memory_ranges() i added in v1 and implement that > > in fdt_enforce_memory_region(). > > There are at most two crash kernel regions, for two crash kernel regions > > case, we cap the memory range [min(regs[*].start), max(regs[*].end)] > > and then remove the memory range in the middle. > > > > v1: > > There are following issues in arm64 kdump: > > 1. We use crashkernel=X to reserve crashkernel below 4G, which > > will fail when there is no enough low memory. > > 2. If reserving crashkernel above 4G, in this case, crash dump > > kernel will boot failure because there is no low memory available > > for allocation. > > > > To solve these issues, change the behavior of crashkernel=X. > > crashkernel=X tries low allocation in DMA zone and fall back to high > > allocation if it fails. > > > > We can also use "crashkernel=X,high" to select a high region above > > DMA zone, which also tries to allocate at least 256M low memory in > > DMA zone automatically and "crashkernel=Y,low" can be used to allocate > > specified size low memory. > > > > When reserving crashkernel in high memory, some low memory is reserved > > for crash dump kernel devices. So there may be two regions reserved for > > crash dump kernel. > > In order to distinct from the high region and make no effect to the use > > of existing kexec-tools, rename the low region as "Crash kernel (low)", > > and pass the low region by reusing DT property > > "linux,usable-memory-range". We made the low memory region as the last > > range of "linux,usable-memory-range" to keep compatibility with existing > > user-space and older kdump kernels. > > > > Besides, we need to modify kexec-tools: > > arm64: support more than one crash kernel regions(see [1]) > > > > Another update is document about DT property 'linux,usable-memory-range': > > schemas: update 'linux,usable-memory-range' node schema(see [2]) > > > > > > [1]: https://www.spinics.net/lists/kexec/msg28226.html > > [2]: https://github.com/robherring/dt-schema/pull/19 > > [v1]: https://lkml.org/lkml/2019/4/2/1174 > > [v2]: https://lkml.org/lkml/2019/4/9/86 > > [v3]: https://lkml.org/lkml/2019/4/9/306 > > [v4]: https://lkml.org/lkml/2019/4/15/273 > > [v5]: https://lkml.org/lkml/2019/5/6/1360 > > [v6]: https://lkml.org/lkml/2019/8/30/142 > > [v7]: https://lkml.org/lkml/2019/12/23/411 > > [v8]: https://lkml.org/lkml/2020/5/21/213 > > [v9]: https://lkml.org/lkml/2020/6/28/73 > > [v10]: https://lkml.org/lkml/2020/7/2/1443 > > [v11]: https://lkml.org/lkml/2020/8/1/150 > > [v12]: https://lkml.org/lkml/2020/9/7/1037 > > [v13]: https://lkml.org/lkml/2020/10/31/34 > > [v14]: https://lkml.org/lkml/2021/1/30/53 > > [v15]: https://lkml.org/lkml/2021/10/19/1405 > > [v16]: https://lkml.org/lkml/2021/11/23/435 > > [v17]: https://lkml.org/lkml/2021/12/10/38 > > [v18]: https://lkml.org/lkml/2021/12/22/424 > > [v19]: https://lkml.org/lkml/2021/12/28/203 > > [v20]: https://lkml.org/lkml/2022/1/24/167 > > [v21]: https://lkml.org/lkml/2022/2/26/350 > > > > Chen Zhou (2): > > arm64: kdump: Reimplement crashkernel=X > > of: fdt: Add memory for devices by DT property > > "linux,usable-memory-range" > > > > Zhen Lei (7): > > kdump: return -ENOENT if required cmdline option does not exist > > arm64: Use insert_resource() to simplify code > > arm64: kdump: Remove some redundant checks in map_mem() > > arm64: kdump: Don't force page-level mappings for memory above 4G > > arm64: kdump: Use page-level mapping for the high memory of > > crashkernel > > arm64: kdump: Try not to use NO_BLOCK_MAPPINGS for memory under 4G > > docs: kdump: Update the crashkernel description for arm64 > > > > .../admin-guide/kernel-parameters.txt | 8 +- > > arch/arm64/include/asm/kexec.h | 2 + > > arch/arm64/kernel/machine_kexec.c | 9 +- > > arch/arm64/kernel/machine_kexec_file.c | 12 +- > > arch/arm64/kernel/setup.c | 17 +- > > arch/arm64/mm/init.c | 204 +++++++++++++++++- > > arch/arm64/mm/mmu.c | 68 +++--- > > drivers/of/fdt.c | 33 ++- > > kernel/crash_core.c | 3 +- > > 9 files changed, 278 insertions(+), 78 deletions(-) > > > > -- > Regards, > Zhen Lei >
On 2022/4/25 10:45, Baoquan He wrote: > On 04/25/22 at 10:19am, Leizhen (ThunderTown) wrote: >> >> Hi, Catalin Marinas and Will Deacon: >> Do you have time to review patches 3-4 and 6-7? These patches are a little more >> relevant to the arm64 architecture. >> >> Hi, Baoquan He: >> Can you review patches 1 and 5 again? > > Yes, and I will test and review the whole series if no one check the > arm64 related changes. Okay, thank you so much. > >> >>
Changes since [v21]: 1. Update the commit message of patch 1 and 5. 2. Add some comments for reserve_crashkernel() in patch 5. Thanks to Baoquan He and John Donnelly for their review comments. Because v5.18-rc1 has added a new patch commit 031495635b46 ("arm64: Do not defer reserve_crashkernel() for platforms with no DMA memory zones") There are many new scenarios: 1) The mappings may be block or page-level. 2) The call to reserve_crashkernel() may or may not be deferred. 3) The the upper limit of DMA address may be 4G, or less than 4G. Or the upper limit of physical memory, because SMMU can do the mapping. The code of patch 1-2, 8-9 keep no change, because the above-mentioned issues are not involved. The code of patch 5 only makes the following changes: - if (crash_base >= SZ_4G) + /* + * When both CONFIG_ZONE_DMA and CONFIG_ZONE_DMA32 are disabled, the + * CRASH_ADDR_LOW_MAX equals the upper limit of physical memory, so + * the 'crash_base' of high memory can not exceed it. To follow the + * description of "crashkernel=X,high" option, add below 'high' + * condition to make sure the crash low memory will be reserved. + */ + if ((crash_base >= CRASH_ADDR_LOW_MAX) || high) { Change SZ_4G to CRASH_ADDR_LOW_MAX, because arm64_dma_phys_limit may be less than 4G or greater than 4G. The check 'high' is used for "crashkernel=X,high" and "(crash_base >= CRASH_ADDR_LOW_MAX)" is used for "crashkernel=X[@offset]". Patch 3-4 to allow block mappings for memory above 4G. Patch 6-7 to support only crash high memory or fixed memory range specified by crashkernel=X@offset use page-level mapping, to allow other areas use block mapping. These four patches are for performance optimization purposes. For details about the technical feasibility analysis, please see the commit messages. Now the implementation of arm64 is very different from that of x86. It's no longer suitable for both of them to share code. Changes since [v20]: 1. Check whether crashkernel=Y,low is incorrectly configured or not configured. Do different processing. 2. Share the existing description of x86. The configuration of arm64 is the same as that of x86. 3. Define the value of macro CRASH_ADDR_HIGH_MAX as memblock.current_limit, instead of MEMBLOCK_ALLOC_ACCESSIBLE. 4. To improve readability, some lightweight code adjustments have been made to reserve_craskernel(), including comments. 5. The defined value of DEFAULT_CRASH_KERNEL_LOW_SIZE reconsiders swiotlb, just like x86, to share documents. Thanks to Baoquan He for his careful review. The test cases are as follows: (Please update the kexec tool to the latest version) 1) crashkernel=4G //high=4G, low=256M 2) crashkernel=4G crashkernel=512M,high crashkernel=512M,low //high=4G, low=256M, high and low are ignored 3) crashkernel=4G crashkernel=512M,high //high=4G, low=256M, high is ignored 4) crashkernel=4G crashkernel=512M,low //high=4G, low=256M, low is ignored 5) crashkernel=4G@0xe0000000 //high=0G, low=0M, cannot allocate, failed 6) crashkernel=512M //high=0G, low=512M 7) crashkernel=128M //high=0G, low=128M 8) crashkernel=512M@0xde000000 //512M@3552M //high=0G, low=512M 9) crashkernel=4G,high //high=4G, low=256M a) crashkernel=4G,high crashkernel=512M,low //high=4G, low=512M b) crashkernel=512M,high crashkernel=128M,low //high=512M, low=128M c) crashkernel=128M,high //high=128M, low=256M d) crashkernel=512M,low //high=0G, low=0M, invalid e) crashkernel=512M,high crashkernel=0,low //high=512M, low=0M f) crashkernel=4G,high crashkernel=ab,low //high=0G, low=0M, invalid Changes since [v19]: 1. Temporarily stop making reserve_crashkernel[_low]() generic. There are a lot of details need to be considered, which can take a long time. Because "make generic" does not add new functions and does not improve performance, maybe I should say it's just a cleanup. So by stripping it out and leaving it for other patches later, we can aggregate the changes to the main functions. 2. Use insert_resource() to replace request_resource(), this not only simplifies the code, but also reduces the differences between arm64 and x86 implementations. 3. As commit 157752d84f5d ("kexec: use Crash kernel for Crash kernel low") do for x86, we can also extend kexec-tools for arm64, and it's currently applied. See: https://www.spinics.net/lists/kexec/msg28284.html Thank you very much, Borislav Petkov, for so many valuable comments. Changes since [v17]: v17 --> v19 1. Patch 0001-0004 Introduce generic parse_crashkernel_high_low() to bring the parsing of "crashkernel=X,high" and the parsing of "crashkernel=X,low" together, then use it instead of the call to parse_crashkernel_{high|low}(). Two confusing parameters of parse_crashkernel_{high|low}() are deleted. I previously sent these four patches separately: [1] https://lkml.org/lkml/2021/12/25/40 2. Patch 0005-0009 Introduce generic reserve_crashkernel_mem[_low](), the implementation of these two functions is based on function reserve_crashkernel[_low]() in arch/x86/kernel/setup.c. There is no functional change for x86. 1) The check position of xen_pv_domain() does not change. 2) Still 1M alignment for crash kernel fixed region, when 'base' is specified. To avoid compilation problems on other architectures: patch 0004 moves the definition of global variable crashk[_low]_res from kexec_core.c to crash_core.c, and provide default definitions for all macros involved, a particular platform can redefine these macros to override the default values. 3. 0010, only one line of comment was changed. 4. 0011 1) crashk_low_res may also a valid reserved memory, should be checked in crash_is_nosave(), see arch/arm64/kernel/machine_kexec. 2) Drop memblock_mark_nomap() for crashk_low_res, because of: 2687275a5843 arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required 3) Also call kmemleak_ignore_phys() for crashk_low_res, because of: 85f58eb18898 arm64: kdump: Skip kmemleak scan reserved memory for kdump 5. 0012, slightly rebased, because the following patch is applied in advance. https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git/commit/?h=dt/linus&id=8347b41748c3019157312fbe7f8a6792ae396eb7 6. 0013, no change. Others: 1. Discard add ARCH_WANT_RESERVE_CRASH_KERNEL 2. When allocating crash low memory, the start address still starts from 0. low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0, CRASH_ADDR_LOW_MAX); 3. Discard change (1ULL << 32) to CRASH_ADDR_LOW_MAX. 4. Ensure the check position of xen_pv_domain() have no change. 5. Except patch 0010 and 0012, all "Tested-by", "Reviewed-by", "Acked-by" are removed. 6. Update description. Changes since [v16] - Because no functional changes in this version, so add "Tested-by: Dave Kleikamp <dave.kleikamp@oracle.com>" for patch 1-9 - Add "Reviewed-by: Rob Herring <robh@kernel.org>" for patch 8 - Update patch 9 based on the review comments of Rob Herring - As Catalin Marinas's suggestion, merge the implementation of ARCH_WANT_RESERVE_CRASH_KERNEL into patch 5. Ensure that the contents of X86 and ARM64 do not overlap, and reduce unnecessary temporary differences. Changes since [v15] - Aggregate the processing of "linux,usable-memory-range" into one function. Only patch 9-10 have been updated. Changes since [v14] - Recovering the requirement that the CrashKernel memory regions on X86 only requires 1 MiB alignment. - Combine patches 5 and 6 in v14 into one. The compilation warning fixed by patch 6 was introduced by patch 5 in v14. - As with crashk_res, crashk_low_res is also processed by crash_exclude_mem_range() in patch 7. - Due to commit b261dba2fdb2 ("arm64: kdump: Remove custom linux,usable-memory-range handling") has removed the architecture-specific code, extend the property "linux,usable-memory-range" in the platform-agnostic FDT core code. See patch 9. - Discard the x86 description update in the document, because the description has been updated by commit b1f4c363666c ("Documentation: kdump: update kdump guide"). - Change "arm64" to "ARM64" in Doc. Changes since [v13] - Rebased on top of 5.11-rc5. - Introduce config CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL. Since reserve_crashkernel[_low]() implementations are quite similar on other architectures, so have CONFIG_ARCH_WANT_RESERVE_CRASH_KERNEL in arch/Kconfig and select this by X86 and ARM64. - Some minor cleanup. Changes since [v12] - Rebased on top of 5.10-rc1. - Keep CRASH_ALIGN as 16M suggested by Dave. - Drop patch "kdump: add threshold for the required memory". - Add Tested-by from John. Changes since [v11] - Rebased on top of 5.9-rc4. - Make the function reserve_crashkernel() of x86 generic. Suggested by Catalin, make the function reserve_crashkernel() of x86 generic and arm64 use the generic version to reimplement crashkernel=X. Changes since [v10] - Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin. Changes since [v9] - Patch 1 add Acked-by from Dave. - Update patch 5 according to Dave's comments. - Update chosen schema. Changes since [v8] - Reuse DT property "linux,usable-memory-range". Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the low memory region. - Fix kdump broken with ZONE_DMA reintroduced. - Update chosen schema. Changes since [v7] - Move x86 CRASH_ALIGN to 2M Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. - Update Documentation/devicetree/bindings/chosen.txt. Add corresponding documentation to Documentation/devicetree/bindings/chosen.txt suggested by Arnd. - Add Tested-by from Jhon and pk. Changes since [v6] - Fix build errors reported by kbuild test robot. Changes since [v5] - Move reserve_crashkernel_low() into kernel/crash_core.c. - Delete crashkernel=X,high. - Modify crashkernel=X,low. If crashkernel=X,low is specified simultaneously, reserve spcified size low memory for crash kdump kernel devices firstly and then reserve memory above 4G. In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then pass to crash dump kernel by DT property "linux,low-memory-range". - Update Documentation/admin-guide/kdump/kdump.rst. Changes since [v4] - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. Changes since [v3] - Add memblock_cap_memory_ranges back for multiple ranges. - Fix some compiling warnings. Changes since [v2] - Split patch "arm64: kdump: support reserving crashkernel above 4G" as two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate patch. Changes since [v1]: - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. - Remove memblock_cap_memory_ranges() i added in v1 and implement that in fdt_enforce_memory_region(). There are at most two crash kernel regions, for two crash kernel regions case, we cap the memory range [min(regs[*].start), max(regs[*].end)] and then remove the memory range in the middle. v1: There are following issues in arm64 kdump: 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail when there is no enough low memory. 2. If reserving crashkernel above 4G, in this case, crash dump kernel will boot failure because there is no low memory available for allocation. To solve these issues, change the behavior of crashkernel=X. crashkernel=X tries low allocation in DMA zone and fall back to high allocation if it fails. We can also use "crashkernel=X,high" to select a high region above DMA zone, which also tries to allocate at least 256M low memory in DMA zone automatically and "crashkernel=Y,low" can be used to allocate specified size low memory. When reserving crashkernel in high memory, some low memory is reserved for crash dump kernel devices. So there may be two regions reserved for crash dump kernel. In order to distinct from the high region and make no effect to the use of existing kexec-tools, rename the low region as "Crash kernel (low)", and pass the low region by reusing DT property "linux,usable-memory-range". We made the low memory region as the last range of "linux,usable-memory-range" to keep compatibility with existing user-space and older kdump kernels. Besides, we need to modify kexec-tools: arm64: support more than one crash kernel regions(see [1]) Another update is document about DT property 'linux,usable-memory-range': schemas: update 'linux,usable-memory-range' node schema(see [2]) [1]: https://www.spinics.net/lists/kexec/msg28226.html [2]: https://github.com/robherring/dt-schema/pull/19 [v1]: https://lkml.org/lkml/2019/4/2/1174 [v2]: https://lkml.org/lkml/2019/4/9/86 [v3]: https://lkml.org/lkml/2019/4/9/306 [v4]: https://lkml.org/lkml/2019/4/15/273 [v5]: https://lkml.org/lkml/2019/5/6/1360 [v6]: https://lkml.org/lkml/2019/8/30/142 [v7]: https://lkml.org/lkml/2019/12/23/411 [v8]: https://lkml.org/lkml/2020/5/21/213 [v9]: https://lkml.org/lkml/2020/6/28/73 [v10]: https://lkml.org/lkml/2020/7/2/1443 [v11]: https://lkml.org/lkml/2020/8/1/150 [v12]: https://lkml.org/lkml/2020/9/7/1037 [v13]: https://lkml.org/lkml/2020/10/31/34 [v14]: https://lkml.org/lkml/2021/1/30/53 [v15]: https://lkml.org/lkml/2021/10/19/1405 [v16]: https://lkml.org/lkml/2021/11/23/435 [v17]: https://lkml.org/lkml/2021/12/10/38 [v18]: https://lkml.org/lkml/2021/12/22/424 [v19]: https://lkml.org/lkml/2021/12/28/203 [v20]: https://lkml.org/lkml/2022/1/24/167 [v21]: https://lkml.org/lkml/2022/2/26/350 Chen Zhou (2): arm64: kdump: Reimplement crashkernel=X of: fdt: Add memory for devices by DT property "linux,usable-memory-range" Zhen Lei (7): kdump: return -ENOENT if required cmdline option does not exist arm64: Use insert_resource() to simplify code arm64: kdump: Remove some redundant checks in map_mem() arm64: kdump: Don't force page-level mappings for memory above 4G arm64: kdump: Use page-level mapping for the high memory of crashkernel arm64: kdump: Try not to use NO_BLOCK_MAPPINGS for memory under 4G docs: kdump: Update the crashkernel description for arm64 .../admin-guide/kernel-parameters.txt | 8 +- arch/arm64/include/asm/kexec.h | 2 + arch/arm64/kernel/machine_kexec.c | 9 +- arch/arm64/kernel/machine_kexec_file.c | 12 +- arch/arm64/kernel/setup.c | 17 +- arch/arm64/mm/init.c | 204 +++++++++++++++++- arch/arm64/mm/mmu.c | 68 +++--- drivers/of/fdt.c | 33 ++- kernel/crash_core.c | 3 +- 9 files changed, 278 insertions(+), 78 deletions(-)