| Message ID | 20190801152439.11363-1-pasha.tatashin@soleen.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | arm64: MMU enabled kexec relocation | expand |
Just a friendly reminder, please send your comments on this series. It's been a week since I sent out these patches, and no feedback yet. Also, I'd appreciate if anyone could test this series on vhe hardware with vhe kernel, it does not look like QEMU can emulate it yet Thank you, Pasha On Thu, Aug 1, 2019 at 11:24 AM Pavel Tatashin <pasha.tatashin@soleen.com> wrote: > > Enable MMU during kexec relocation in order to improve reboot performance. > > If kexec functionality is used for a fast system update, with a minimal > downtime, the relocation of kernel + initramfs takes a significant portion > of reboot. > > The reason for slow relocation is because it is done without MMU, and thus > not benefiting from D-Cache. > > Performance data > ---------------- > For this experiment, the size of kernel plus initramfs is small, only 25M. > If initramfs was larger, than the improvements would be greater, as time > spent in relocation is proportional to the size of relocation. > > Previously: > kernel shutdown 0.022131328s > relocation 0.440510736s > kernel startup 0.294706768s > > Relocation was taking: 58.2% of reboot time > > Now: > kernel shutdown 0.032066576s > relocation 0.022158152s > kernel startup 0.296055880s > > Now: Relocation takes 6.3% of reboot time > > Total reboot is x2.16 times faster. > > Previous approaches and discussions > ----------------------------------- > https://lore.kernel.org/lkml/20190709182014.16052-1-pasha.tatashin@soleen.com > reserve space for kexec to avoid relocation, involves changes to generic code > to optimize a problem that exists on arm64 only: > > https://lore.kernel.org/lkml/20190716165641.6990-1-pasha.tatashin@soleen.com > The first attempt to enable MMU, some bugs that prevented performance > improvement. The page tables unnecessary configured idmap for the whole > physical space. > > https://lore.kernel.org/lkml/20190731153857.4045-1-pasha.tatashin@soleen.com > No linear copy, bug with EL2 reboots. > > Pavel Tatashin (8): > kexec: quiet down kexec reboot > arm64, mm: transitional tables > arm64: hibernate: switch to transtional page tables. > kexec: add machine_kexec_post_load() > arm64, kexec: move relocation function setup and clean up > arm64, kexec: add expandable argument to relocation function > arm64, kexec: configure transitional page table for kexec > arm64, kexec: enable MMU during kexec relocation > > arch/arm64/Kconfig | 4 + > arch/arm64/include/asm/kexec.h | 51 ++++- > arch/arm64/include/asm/pgtable-hwdef.h | 1 + > arch/arm64/include/asm/trans_table.h | 68 ++++++ > arch/arm64/kernel/asm-offsets.c | 14 ++ > arch/arm64/kernel/cpu-reset.S | 4 +- > arch/arm64/kernel/cpu-reset.h | 8 +- > arch/arm64/kernel/hibernate.c | 261 ++++++----------------- > arch/arm64/kernel/machine_kexec.c | 199 ++++++++++++++---- > arch/arm64/kernel/relocate_kernel.S | 196 +++++++++--------- > arch/arm64/mm/Makefile | 1 + > arch/arm64/mm/trans_table.c | 273 +++++++++++++++++++++++++ > kernel/kexec.c | 4 + > kernel/kexec_core.c | 8 +- > kernel/kexec_file.c | 4 + > kernel/kexec_internal.h | 2 + > 16 files changed, 758 insertions(+), 340 deletions(-) > create mode 100644 arch/arm64/include/asm/trans_table.h > create mode 100644 arch/arm64/mm/trans_table.c > > -- > 2.22.0 >
Hi, It is been two weeks, and no review activity yet. Please help with reviewing this work. Thank you, Pasha On Thu, Aug 8, 2019 at 2:44 PM Pavel Tatashin <pasha.tatashin@soleen.com> wrote: > > Just a friendly reminder, please send your comments on this series. > It's been a week since I sent out these patches, and no feedback yet. > Also, I'd appreciate if anyone could test this series on vhe hardware > with vhe kernel, it does not look like QEMU can emulate it yet > > Thank you, > Pasha > > On Thu, Aug 1, 2019 at 11:24 AM Pavel Tatashin > <pasha.tatashin@soleen.com> wrote: > > > > Enable MMU during kexec relocation in order to improve reboot performance. > > > > If kexec functionality is used for a fast system update, with a minimal > > downtime, the relocation of kernel + initramfs takes a significant portion > > of reboot. > > > > The reason for slow relocation is because it is done without MMU, and thus > > not benefiting from D-Cache. > > > > Performance data > > ---------------- > > For this experiment, the size of kernel plus initramfs is small, only 25M. > > If initramfs was larger, than the improvements would be greater, as time > > spent in relocation is proportional to the size of relocation. > > > > Previously: > > kernel shutdown 0.022131328s > > relocation 0.440510736s > > kernel startup 0.294706768s > > > > Relocation was taking: 58.2% of reboot time > > > > Now: > > kernel shutdown 0.032066576s > > relocation 0.022158152s > > kernel startup 0.296055880s > > > > Now: Relocation takes 6.3% of reboot time > > > > Total reboot is x2.16 times faster. > > > > Previous approaches and discussions > > ----------------------------------- > > https://lore.kernel.org/lkml/20190709182014.16052-1-pasha.tatashin@soleen.com > > reserve space for kexec to avoid relocation, involves changes to generic code > > to optimize a problem that exists on arm64 only: > > > > https://lore.kernel.org/lkml/20190716165641.6990-1-pasha.tatashin@soleen.com > > The first attempt to enable MMU, some bugs that prevented performance > > improvement. The page tables unnecessary configured idmap for the whole > > physical space. > > > > https://lore.kernel.org/lkml/20190731153857.4045-1-pasha.tatashin@soleen.com > > No linear copy, bug with EL2 reboots. > > > > Pavel Tatashin (8): > > kexec: quiet down kexec reboot > > arm64, mm: transitional tables > > arm64: hibernate: switch to transtional page tables. > > kexec: add machine_kexec_post_load() > > arm64, kexec: move relocation function setup and clean up > > arm64, kexec: add expandable argument to relocation function > > arm64, kexec: configure transitional page table for kexec > > arm64, kexec: enable MMU during kexec relocation > > > > arch/arm64/Kconfig | 4 + > > arch/arm64/include/asm/kexec.h | 51 ++++- > > arch/arm64/include/asm/pgtable-hwdef.h | 1 + > > arch/arm64/include/asm/trans_table.h | 68 ++++++ > > arch/arm64/kernel/asm-offsets.c | 14 ++ > > arch/arm64/kernel/cpu-reset.S | 4 +- > > arch/arm64/kernel/cpu-reset.h | 8 +- > > arch/arm64/kernel/hibernate.c | 261 ++++++----------------- > > arch/arm64/kernel/machine_kexec.c | 199 ++++++++++++++---- > > arch/arm64/kernel/relocate_kernel.S | 196 +++++++++--------- > > arch/arm64/mm/Makefile | 1 + > > arch/arm64/mm/trans_table.c | 273 +++++++++++++++++++++++++ > > kernel/kexec.c | 4 + > > kernel/kexec_core.c | 8 +- > > kernel/kexec_file.c | 4 + > > kernel/kexec_internal.h | 2 + > > 16 files changed, 758 insertions(+), 340 deletions(-) > > create mode 100644 arch/arm64/include/asm/trans_table.h > > create mode 100644 arch/arm64/mm/trans_table.c > > > > -- > > 2.22.0 > >
Hi Pavel, On 08/08/2019 19:44, Pavel Tatashin wrote: > Just a friendly reminder, please send your comments on this series. (Please don't top-post) > It's been a week since I sent out these patches, and no feedback yet. A week is not a lot of time, people are busy, go to conferences, some even dare to take holiday! > Also, I'd appreciate if anyone could test this series on vhe hardware > with vhe kernel, it does not look like QEMU can emulate it yet This locks up during resume from hibernate on my AMD Seattle, a regular v8.0 machine. Please try and build the series to reduce review time. What you have here is an all-new page-table generation API, which you switch hibernate and kexec too. This is effectively a new implementation of hibernate and kexec. There are three things here that need review. You have a regression in your all-new implementation of hibernate. It took six months (and lots of review) to get the existing code right, please don't rip it out if there is nothing wrong with it. Instead, please just move the hibernate copy_page_tables() code, and then wire kexec up. You shouldn't need to change anything in the copy_page_tables() code as the linear map is the same in both cases. It looks like you are creating the page tables just after the kexec:segments have been loaded. This will go horribly wrong if anything changes between then and kexec time. (e.g. memory you've got mapped gets hot-removed). This needs to be done as late as possible, so we don't waste memory, and the world can't change around us. Reboot notifiers run before kexec, can't we do the memory-allocation there? > On Thu, Aug 1, 2019 at 11:24 AM Pavel Tatashin > <pasha.tatashin@soleen.com> wrote: >> >> Enable MMU during kexec relocation in order to improve reboot performance. >> >> If kexec functionality is used for a fast system update, with a minimal >> downtime, the relocation of kernel + initramfs takes a significant portion >> of reboot. >> >> The reason for slow relocation is because it is done without MMU, and thus >> not benefiting from D-Cache. >> >> Performance data >> ---------------- >> For this experiment, the size of kernel plus initramfs is small, only 25M. >> If initramfs was larger, than the improvements would be greater, as time >> spent in relocation is proportional to the size of relocation. >> >> Previously: >> kernel shutdown 0.022131328s >> relocation 0.440510736s >> kernel startup 0.294706768s >> >> Relocation was taking: 58.2% of reboot time >> >> Now: >> kernel shutdown 0.032066576s >> relocation 0.022158152s >> kernel startup 0.296055880s >> >> Now: Relocation takes 6.3% of reboot time >> >> Total reboot is x2.16 times faster. When I first saw these numbers they were ~'0.29s', which I wrongly assumed was 29 seconds. Savings in milliseconds, for _reboot_ is a hard sell. I'm hoping that on the machines that take minutes to kexec we'll get numbers that make this change more convincing. Thanks, James
Hi James, Thank you for your feedback. My replies below: > > Also, I'd appreciate if anyone could test this series on vhe hardware > > with vhe kernel, it does not look like QEMU can emulate it yet > > This locks up during resume from hibernate on my AMD Seattle, a regular v8.0 machine. Thanks for reporting a bug I will root cause and fix it. > Please try and build the series to reduce review time. What you have here is an all-new > page-table generation API, which you switch hibernate and kexec too. This is effectively a > new implementation of hibernate and kexec. There are three things here that need review. > > You have a regression in your all-new implementation of hibernate. It took six months (and > lots of review) to get the existing code right, please don't rip it out if there is > nothing wrong with it. > Instead, please just move the hibernate copy_page_tables() code, and then wire kexec up. > You shouldn't need to change anything in the copy_page_tables() code as the linear map is > the same in both cases. It is not really an all-new implementation of hibernate (for kexec it is true though). I used the current implementation of hibernate as bases, and simply generalized the functions by providing a flexible interface. So what you are asking is actually exactly what I am doing. I realize, that I introduced a bug that I will fix. > It looks like you are creating the page tables just after the kexec:segments have been > loaded. This will go horribly wrong if anything changes between then and kexec time. (e.g. > memory you've got mapped gets hot-removed). > This needs to be done as late as possible, so we don't waste memory, and the world can't > change around us. Reboot notifiers run before kexec, can't we do the memory-allocation there? Kexec by design does not allow allocate during kexec time. This is because we cannot fail during kexec syscall. All allocations must be done during kexec load time. Kernel memory cannot be hot-removed, as it is not part of ZONE_MOVABLE, and cannot be migrated. The current implementation relies on this assumption as well: during load time the (struct kimage) -> head contains the physical addresses of sources and destinations. If sources can be moved, this array will be broken. > >> Previously: > >> kernel shutdown 0.022131328s > >> relocation 0.440510736s > >> kernel startup 0.294706768s > >> > >> Relocation was taking: 58.2% of reboot time > >> > >> Now: > >> kernel shutdown 0.032066576s > >> relocation 0.022158152s > >> kernel startup 0.296055880s > >> > >> Now: Relocation takes 6.3% of reboot time > >> > >> Total reboot is x2.16 times faster. > > When I first saw these numbers they were ~'0.29s', which I wrongly assumed was 29 seconds. > Savings in milliseconds, for _reboot_ is a hard sell. I'm hoping that on the machines that > take minutes to kexec we'll get numbers that make this change more convincing. Sure, this userland is very small kernel+userland is only 47M. Here is another data point: fitImage: 380M, it contains a larger userland. The numbers for kernel shutdown and startup are the same as this is the same kernel, but relocation takes: 3.58s shutdown: 0.02s relocation: 3.58s startup: 0.30s Relocation take 88% of reboot time. And, we must have it under one second. Thank you, Pasha
Hi Pavel, On 15/08/2019 21:09, Pavel Tatashin wrote: >>> Also, I'd appreciate if anyone could test this series on vhe hardware >>> with vhe kernel, it does not look like QEMU can emulate it yet >> >> This locks up during resume from hibernate on my AMD Seattle, a regular v8.0 machine. > > Thanks for reporting a bug I will root cause and fix it. >> Please try and build the series to reduce review time. What you have here is an all-new >> page-table generation API, which you switch hibernate and kexec too. This is effectively a >> new implementation of hibernate and kexec. There are three things here that need review. >> >> You have a regression in your all-new implementation of hibernate. It took six months (and >> lots of review) to get the existing code right, please don't rip it out if there is >> nothing wrong with it. > >> Instead, please just move the hibernate copy_page_tables() code, and then wire kexec up. >> You shouldn't need to change anything in the copy_page_tables() code as the linear map is >> the same in both cases. > It is not really an all-new implementation of hibernate (for kexec it > is true though). I used the current implementation of hibernate as > bases, and simply generalized the functions by providing a flexible > interface. So what you are asking is actually exactly what I am doing. I disagree. The resume page-table code is the bulk of the complexity in hibernate.c. Your first patch dumps ~200 lines of differently-complex code, and your second switches hibernate over to it. Instead, please move that code, keeping it as it is. git will spot the move, and the generated diffstat should only reflect the build-system changes. You don't need to 'switch hibernate to transitional page tables.' Adding kexec will then show-up what needs changing, each change comes with a commit message explaining why. Having these as 'generalisations' in the first patch is a mess. There is existing code that we don't want to break. Any changes need to be done as a sequence of small incremental changes. It can't be reviewed any other way. > I realize, that I introduced a bug that I will fix. Done as a sequence of small incremental changes, I could bisect it to the patch that introduces the bug, and probably fix it from the description in the commit message. >> It looks like you are creating the page tables just after the kexec:segments have been >> loaded. This will go horribly wrong if anything changes between then and kexec time. (e.g. >> memory you've got mapped gets hot-removed). >> This needs to be done as late as possible, so we don't waste memory, and the world can't >> change around us. Reboot notifiers run before kexec, can't we do the memory-allocation there? > Kexec by design does not allow allocate during kexec time. This is > because we cannot fail during kexec syscall. This problem needs solving. | Reboot notifiers run before kexec, can't we do the memory-allocation there? > All allocations must be done during kexec load time. This increases the memory footprint. I don't think we should waste ~2MB per GB of kernel memory on this feature. (Assuming 4K pages and rodata_full) Another option is to allocate this memory at load time, but then free it so it can be used in the meantime. You can keep the list of allocated pfn, as we know they aren't in use by the running kernel, kexec metadata, loaded images etc. Memory hotplug would need handling carefully, as would anything that 'donates' memory to another agent. (I suspect the TEE stuff does this, I don't know how it interacts with kexec) > Kernel memory cannot be hot-removed, as > it is not part of ZONE_MOVABLE, and cannot be migrated. Today, yes. Tomorrow?, "arm64/mm: Enable memory hot remove": https://lore.kernel.org/r/1563171470-3117-1-git-send-email-anshuman.khandual@arm.com >>>> Previously: >>>> kernel shutdown 0.022131328s >>>> relocation 0.440510736s >>>> kernel startup 0.294706768s >>>> >>>> Relocation was taking: 58.2% of reboot time >>>> >>>> Now: >>>> kernel shutdown 0.032066576s >>>> relocation 0.022158152s >>>> kernel startup 0.296055880s >>>> >>>> Now: Relocation takes 6.3% of reboot time >>>> >>>> Total reboot is x2.16 times faster. >> >> When I first saw these numbers they were ~'0.29s', which I wrongly assumed was 29 seconds. >> Savings in milliseconds, for _reboot_ is a hard sell. I'm hoping that on the machines that >> take minutes to kexec we'll get numbers that make this change more convincing. > Sure, this userland is very small kernel+userland is only 47M. Here is > another data point: fitImage: 380M, it contains a larger userland. > The numbers for kernel shutdown and startup are the same as this is > the same kernel, but relocation takes: 3.58s > shutdown: 0.02s > relocation: 3.58s > startup: 0.30s > > Relocation take 88% of reboot time. And, we must have it under one second. Where does this one second number come from? (was it ever a reasonable starting point?) Thanks, James
Hi James, Thank you for your feedback, my replies below: > > It is not really an all-new implementation of hibernate (for kexec it > > is true though). I used the current implementation of hibernate as > > bases, and simply generalized the functions by providing a flexible > > interface. So what you are asking is actually exactly what I am doing. > > I disagree. The resume page-table code is the bulk of the complexity in hibernate.c. Your > first patch dumps ~200 lines of differently-complex code, and your second switches > hibernate over to it. OK, I will make the change incremental. > > Instead, please move that code, keeping it as it is. git will spot the move, and the > generated diffstat should only reflect the build-system changes. You don't need to 'switch > hibernate to transitional page tables.' > > Adding kexec will then show-up what needs changing, each change comes with a commit > message explaining why. Having these as 'generalisations' in the first patch is a mess. Makes sense, I will fix it. > > There is existing code that we don't want to break. Any changes need to be done as a > sequence of small incremental changes. It can't be reviewed any other way. > > > > I realize, that I introduced a bug that I will fix. > > Done as a sequence of small incremental changes, I could bisect it to the patch that > introduces the bug, and probably fix it from the description in the commit message. BTW, I root caused it, there were two trivial errors: 1. In "arm64, mm: transitional tables" int i = pgd_index(addr); In trans_table_copy_*: should be: pte_index(), pmd_index(), pud_index(), accordingly. 2. In trans_table_create_copy() pgd_offset_k(PAGE_OFFSET) should be: mm_init.pgd > >> It looks like you are creating the page tables just after the kexec:segments have been > >> loaded. This will go horribly wrong if anything changes between then and kexec time. (e.g. > >> memory you've got mapped gets hot-removed). > >> This needs to be done as late as possible, so we don't waste memory, and the world can't > >> change around us. Reboot notifiers run before kexec, can't we do the memory-allocation there? > > > Kexec by design does not allow allocate during kexec time. This is > > because we cannot fail during kexec syscall. > > This problem needs solving. > > | Reboot notifiers run before kexec, can't we do the memory-allocation there? > > > > All allocations must be done during kexec load time. > > This increases the memory footprint. I don't think we should waste ~2MB per GB of kernel > memory on this feature. (Assuming 4K pages and rodata_full) > > Another option is to allocate this memory at load time, but then free it so it can be used > in the meantime. You can keep the list of allocated pfn, as we know they aren't in use by > the running kernel, kexec metadata, loaded images etc. This is until a new kernel module is loaded, I do not think this is safe to do. In my opinion 2M per 1 GB is a fair trade off for a faster kexec performance. Unlike with crash kexec for which we do not add any memory useage, the kernel does not have to be all the time in memory, but can be loaded by user before reboot. If machine is so scare on memory resources that 2M per 1G matters, user simply won't keep new kernel in memory until it is actually needed. > > Memory hotplug would need handling carefully, as would anything that 'donates' memory to > another agent. (I suspect the TEE stuff does this, I don't know how it interacts with kexec) > > > > Kernel memory cannot be hot-removed, as > > it is not part of ZONE_MOVABLE, and cannot be migrated. > > Today, yes. Tomorrow?, "arm64/mm: Enable memory hot remove": > https://lore.kernel.org/r/1563171470-3117-1-git-send-email-anshuman.khandual@arm.com I understand that ARM64 is about to get hot-remove feature, but what I am saying is that my feature does not introduce new problem because the current kexec code assumes that kernel memory is not movable (array of sparse physical source dest addresses in kimage->head). It is possible to offline and hot-remove only memory that can be freed by page migration, the pages that were allocated for kexec kernel are not one of them. > >>>> Previously: > >>>> kernel shutdown 0.022131328s > >>>> relocation 0.440510736s > >>>> kernel startup 0.294706768s > >>>> > >>>> Relocation was taking: 58.2% of reboot time > >>>> > >>>> Now: > >>>> kernel shutdown 0.032066576s > >>>> relocation 0.022158152s > >>>> kernel startup 0.296055880s > >>>> > >>>> Now: Relocation takes 6.3% of reboot time > >>>> > >>>> Total reboot is x2.16 times faster. > >> > >> When I first saw these numbers they were ~'0.29s', which I wrongly assumed was 29 seconds. > >> Savings in milliseconds, for _reboot_ is a hard sell. I'm hoping that on the machines that > >> take minutes to kexec we'll get numbers that make this change more convincing. > > > Sure, this userland is very small kernel+userland is only 47M. Here is > > another data point: fitImage: 380M, it contains a larger userland. > > The numbers for kernel shutdown and startup are the same as this is > > the same kernel, but relocation takes: 3.58s > > shutdown: 0.02s > > relocation: 3.58s > > startup: 0.30s > > > > Relocation take 88% of reboot time. And, we must have it under one second. > > Where does this one second number come from? (was it ever a reasonable starting point?) Currently we have two fitImages for this system in development: one that has a bare minimal userland, only ~40 packages, and another has a more complete userland. So, my first experiment shows the data from this first bare minimum ftImage, the second experiment from the second more complete fitImage. As I stated in cover letter, kexec time is proportional to the size of the image and this series fixes this scalability issue by making relocation ~20 times faster. Pasha