| Message ID | 20230912141549.278777-63-ardb@google.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | arm64: Add support for LPA2 at stage1 and WXN | expand |
On Tue, 12 Sep 2023 14:15:50 +0000, Ard Biesheuvel wrote: > This is a followup to [0], which was sent out more than 6 months ago. > Thanks to Ryan and Mark for feedback and review. This series is > independent from Ryan's work on adding support for LPA2 to KVM - the > only potential source of conflict should be the patch "arm64: kvm: Limit > HYP VA and host S2 range to 48 bits when LPA2 is in effect", which could > simply be dropped in favour of the KVM changes to make it support LPA2. > > [...] I pushed the series to the arm64 for-next/lpa2-stage1 branch. If something falls apart badly in -next (other than the typical conflicts), I can drop the series before the upcoming merging window. There are a couple of patches touching KVM, it would be good to get an ack from Marc or Oliver (I'll rebase the branch if you do but no worries if you don't get around). I think Ard's C++ style comments will disappear with Ryan's LPA2 support for stage 2 (whenever that will get merged). https://lore.kernel.org/r/20230912141549.278777-119-ardb@google.com https://lore.kernel.org/r/20230912141549.278777-120-ardb@google.com Talking of KVM, we'll get a conflict with next (depending on which branch is picked first by sfr, the polarity may differ). That's my resolution of merging Ard's patches into -next: diff --cc arch/arm64/kvm/hyp/nvhe/mem_protect.c index 8d0a5834e883,c20b08cf1f03..34c17ec521c7 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@@ -128,9 -128,11 +128,11 @@@ static void prepare_host_vtcr(void /* The host stage 2 is id-mapped, so use parange for T0SZ */ parange = kvm_get_parange(id_aa64mmfr0_el1_sys_val); phys_shift = id_aa64mmfr0_parange_to_phys_shift(parange); + if (IS_ENABLED(CONFIG_ARM64_LPA2) && phys_shift > 48) + phys_shift = 48; // not implemented yet - host_mmu.arch.vtcr = kvm_get_vtcr(id_aa64mmfr0_el1_sys_val, - id_aa64mmfr1_el1_sys_val, phys_shift); + host_mmu.arch.mmu.vtcr = kvm_get_vtcr(id_aa64mmfr0_el1_sys_val, + id_aa64mmfr1_el1_sys_val, phys_shift); } static bool host_stage2_force_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot prot); So Marc, Oliver, if you want to avoid this, you could merge the lpa2-stage1 branch into the KVM tree once I freeze it. Thanks.
On Mon, 23 Oct 2023 18:45:25 +0100, Catalin Marinas <catalin.marinas@arm.com> wrote: > > On Tue, 12 Sep 2023 14:15:50 +0000, Ard Biesheuvel wrote: > > This is a followup to [0], which was sent out more than 6 months ago. > > Thanks to Ryan and Mark for feedback and review. This series is > > independent from Ryan's work on adding support for LPA2 to KVM - the > > only potential source of conflict should be the patch "arm64: kvm: Limit > > HYP VA and host S2 range to 48 bits when LPA2 is in effect", which could > > simply be dropped in favour of the KVM changes to make it support LPA2. > > > > [...] > > I pushed the series to the arm64 for-next/lpa2-stage1 branch. If > something falls apart badly in -next (other than the typical conflicts), > I can drop the series before the upcoming merging window. > > There are a couple of patches touching KVM, it would be good to get an > ack from Marc or Oliver (I'll rebase the branch if you do but no worries > if you don't get around). I think Ard's C++ style comments will > disappear with Ryan's LPA2 support for stage 2 (whenever that will get > merged). > > https://lore.kernel.org/r/20230912141549.278777-119-ardb@google.com > https://lore.kernel.org/r/20230912141549.278777-120-ardb@google.com > > Talking of KVM, we'll get a conflict with next (depending on which > branch is picked first by sfr, the polarity may differ). That's my > resolution of merging Ard's patches into -next: > > diff --cc arch/arm64/kvm/hyp/nvhe/mem_protect.c > index 8d0a5834e883,c20b08cf1f03..34c17ec521c7 > --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c > +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c > @@@ -128,9 -128,11 +128,11 @@@ static void prepare_host_vtcr(void > /* The host stage 2 is id-mapped, so use parange for T0SZ */ > parange = kvm_get_parange(id_aa64mmfr0_el1_sys_val); > phys_shift = id_aa64mmfr0_parange_to_phys_shift(parange); > + if (IS_ENABLED(CONFIG_ARM64_LPA2) && phys_shift > 48) > + phys_shift = 48; // not implemented yet > > - host_mmu.arch.vtcr = kvm_get_vtcr(id_aa64mmfr0_el1_sys_val, > - id_aa64mmfr1_el1_sys_val, phys_shift); > + host_mmu.arch.mmu.vtcr = kvm_get_vtcr(id_aa64mmfr0_el1_sys_val, > + id_aa64mmfr1_el1_sys_val, phys_shift); > } > > static bool host_stage2_force_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot prot); > > So Marc, Oliver, if you want to avoid this, you could merge the > lpa2-stage1 branch into the KVM tree once I freeze it. Yeah, that's probably best (though this looks pretty minor). I'll let Oliver decide on it, as he's in charge this time around. Thanks, M.
On Mon, Oct 23, 2023 at 07:54:10PM +0100, Marc Zyngier wrote: > On Mon, 23 Oct 2023 18:45:25 +0100, > Catalin Marinas <catalin.marinas@arm.com> wrote: > > > > On Tue, 12 Sep 2023 14:15:50 +0000, Ard Biesheuvel wrote: > > > This is a followup to [0], which was sent out more than 6 months ago. > > > Thanks to Ryan and Mark for feedback and review. This series is > > > independent from Ryan's work on adding support for LPA2 to KVM - the > > > only potential source of conflict should be the patch "arm64: kvm: Limit > > > HYP VA and host S2 range to 48 bits when LPA2 is in effect", which could > > > simply be dropped in favour of the KVM changes to make it support LPA2. > > > > > > [...] > > > > I pushed the series to the arm64 for-next/lpa2-stage1 branch. If > > something falls apart badly in -next (other than the typical conflicts), > > I can drop the series before the upcoming merging window. > > > > There are a couple of patches touching KVM, it would be good to get an > > ack from Marc or Oliver (I'll rebase the branch if you do but no worries > > if you don't get around). I think Ard's C++ style comments will > > disappear with Ryan's LPA2 support for stage 2 (whenever that will get > > merged). > > > > https://lore.kernel.org/r/20230912141549.278777-119-ardb@google.com > > https://lore.kernel.org/r/20230912141549.278777-120-ardb@google.com > > > > Talking of KVM, we'll get a conflict with next (depending on which > > branch is picked first by sfr, the polarity may differ). That's my > > resolution of merging Ard's patches into -next: > > > > diff --cc arch/arm64/kvm/hyp/nvhe/mem_protect.c > > index 8d0a5834e883,c20b08cf1f03..34c17ec521c7 > > --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c > > +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c > > @@@ -128,9 -128,11 +128,11 @@@ static void prepare_host_vtcr(void > > /* The host stage 2 is id-mapped, so use parange for T0SZ */ > > parange = kvm_get_parange(id_aa64mmfr0_el1_sys_val); > > phys_shift = id_aa64mmfr0_parange_to_phys_shift(parange); > > + if (IS_ENABLED(CONFIG_ARM64_LPA2) && phys_shift > 48) > > + phys_shift = 48; // not implemented yet > > > > - host_mmu.arch.vtcr = kvm_get_vtcr(id_aa64mmfr0_el1_sys_val, > > - id_aa64mmfr1_el1_sys_val, phys_shift); > > + host_mmu.arch.mmu.vtcr = kvm_get_vtcr(id_aa64mmfr0_el1_sys_val, > > + id_aa64mmfr1_el1_sys_val, phys_shift); > > } > > > > static bool host_stage2_force_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot prot); > > > > So Marc, Oliver, if you want to avoid this, you could merge the > > lpa2-stage1 branch into the KVM tree once I freeze it. > > Yeah, that's probably best (though this looks pretty minor). I'll let > Oliver decide on it, as he's in charge this time around. For some reason git manages to resolve it now without any problem. Not sure what happened when I tried this last week. Let's see if Stephen reports any conflicts. Thanks for the acks.
On Mon, Oct 23, 2023 at 09:44:41PM +0100, Catalin Marinas wrote: [...] > > > So Marc, Oliver, if you want to avoid this, you could merge the > > > lpa2-stage1 branch into the KVM tree once I freeze it. > > > > Yeah, that's probably best (though this looks pretty minor). I'll let > > Oliver decide on it, as he's in charge this time around. > > For some reason git manages to resolve it now without any problem. Not > sure what happened when I tried this last week. Let's see if Stephen > reports any conflicts. I accidentally dropped Marc's vtcr patch, which accounts for some of the conflicts. Looks like Stephen did hit a conflict after all, I may as well pull the branch once it is stable.
On 23/10/2023 18:45, Catalin Marinas wrote: > On Tue, 12 Sep 2023 14:15:50 +0000, Ard Biesheuvel wrote: >> This is a followup to [0], which was sent out more than 6 months ago. >> Thanks to Ryan and Mark for feedback and review. This series is >> independent from Ryan's work on adding support for LPA2 to KVM - the >> only potential source of conflict should be the patch "arm64: kvm: Limit >> HYP VA and host S2 range to 48 bits when LPA2 is in effect", which could >> simply be dropped in favour of the KVM changes to make it support LPA2. >> >> [...] > > I pushed the series to the arm64 for-next/lpa2-stage1 branch. If > something falls apart badly in -next (other than the typical conflicts), > I can drop the series before the upcoming merging window. > > There are a couple of patches touching KVM, it would be good to get an > ack from Marc or Oliver (I'll rebase the branch if you do but no worries > if you don't get around). I think Ard's C++ style comments will > disappear with Ryan's LPA2 support for stage 2 (whenever that will get > merged). > > https://lore.kernel.org/r/20230912141549.278777-119-ardb@google.com > https://lore.kernel.org/r/20230912141549.278777-120-ardb@google.com I'm planning to rebase the KVM LPA2 series on v6.7-rc1, so if Ard's series gets into that, I can smooth it all out and ensure both series are working together properly. Thanks, Ryan
Hi Oliver, On Tue, Oct 24, 2023 at 06:54:24AM +0000, Oliver Upton wrote: > On Mon, Oct 23, 2023 at 09:44:41PM +0100, Catalin Marinas wrote: > > > > So Marc, Oliver, if you want to avoid this, you could merge the > > > > lpa2-stage1 branch into the KVM tree once I freeze it. > > > > > > Yeah, that's probably best (though this looks pretty minor). I'll let > > > Oliver decide on it, as he's in charge this time around. > > > > For some reason git manages to resolve it now without any problem. Not > > sure what happened when I tried this last week. Let's see if Stephen > > reports any conflicts. > > I accidentally dropped Marc's vtcr patch, which accounts for some of the > conflicts. Looks like Stephen did hit a conflict after all, I may as > well pull the branch once it is stable. The arm64 for-next/lpa2-stage1 branch is stable now. I may add stuff on top but wouldn't be rebased. I pushed a fix and some minor cleanups on top since yesterday together with your (and Marc's) Ack for the kvm bits. If you see any conflicts, feel free to merge it into the kvm-arm tree. Thanks.
Hi Oliver, On Tue, Oct 24, 2023 at 05:42:52PM +0100, Catalin Marinas wrote: > On Tue, Oct 24, 2023 at 06:54:24AM +0000, Oliver Upton wrote: > > On Mon, Oct 23, 2023 at 09:44:41PM +0100, Catalin Marinas wrote: > > > > > So Marc, Oliver, if you want to avoid this, you could merge the > > > > > lpa2-stage1 branch into the KVM tree once I freeze it. > > > > > > > > Yeah, that's probably best (though this looks pretty minor). I'll let > > > > Oliver decide on it, as he's in charge this time around. > > > > > > For some reason git manages to resolve it now without any problem. Not > > > sure what happened when I tried this last week. Let's see if Stephen > > > reports any conflicts. > > > > I accidentally dropped Marc's vtcr patch, which accounts for some of the > > conflicts. Looks like Stephen did hit a conflict after all, I may as > > well pull the branch once it is stable. > > The arm64 for-next/lpa2-stage1 branch is stable now. I may add stuff on > top but wouldn't be rebased. I pushed a fix and some minor cleanups on > top since yesterday together with your (and Marc's) Ack for the kvm > bits. If you see any conflicts, feel free to merge it into the kvm-arm > tree. If you haven't pulled this branch already, just hold a bit as we've seen some CI reports since last night. Not sure what caused this but Ard's series was the main change.
On Wed, 25 Oct 2023 at 15:19, Catalin Marinas <catalin.marinas@arm.com> wrote: > > Hi Oliver, > > On Tue, Oct 24, 2023 at 05:42:52PM +0100, Catalin Marinas wrote: > > On Tue, Oct 24, 2023 at 06:54:24AM +0000, Oliver Upton wrote: > > > On Mon, Oct 23, 2023 at 09:44:41PM +0100, Catalin Marinas wrote: > > > > > > So Marc, Oliver, if you want to avoid this, you could merge the > > > > > > lpa2-stage1 branch into the KVM tree once I freeze it. > > > > > > > > > > Yeah, that's probably best (though this looks pretty minor). I'll let > > > > > Oliver decide on it, as he's in charge this time around. > > > > > > > > For some reason git manages to resolve it now without any problem. Not > > > > sure what happened when I tried this last week. Let's see if Stephen > > > > reports any conflicts. > > > > > > I accidentally dropped Marc's vtcr patch, which accounts for some of the > > > conflicts. Looks like Stephen did hit a conflict after all, I may as > > > well pull the branch once it is stable. > > > > The arm64 for-next/lpa2-stage1 branch is stable now. I may add stuff on > > top but wouldn't be rebased. I pushed a fix and some minor cleanups on > > top since yesterday together with your (and Marc's) Ack for the kvm > > bits. If you see any conflicts, feel free to merge it into the kvm-arm > > tree. > > If you haven't pulled this branch already, just hold a bit as we've seen > some CI reports since last night. Not sure what caused this but Ard's > series was the main change. > Please let me know when you have any reproducers.
Hi Catalin, On Wed, Oct 25, 2023 at 02:19:50PM +0100, Catalin Marinas wrote: > If you haven't pulled this branch already, just hold a bit as we've seen > some CI reports since last night. Not sure what caused this but Ard's > series was the main change. I did wind up pulling it, but the merge is sitting on the top of kvmarm/next. Easy enough to roll that back.
On 12/09/2023 15:15, Ard Biesheuvel wrote: > From: Ard Biesheuvel <ardb@kernel.org> > > This is a followup to [0], which was sent out more than 6 months ago. > Thanks to Ryan and Mark for feedback and review. This series is > independent from Ryan's work on adding support for LPA2 to KVM - the > only potential source of conflict should be the patch "arm64: kvm: Limit > HYP VA and host S2 range to 48 bits when LPA2 is in effect", which could > simply be dropped in favour of the KVM changes to make it support LPA2. > > Changes since v3: > - add some acks and incorporate some minor suggested tweaks, mostly > related to coding style and comments > - rebase onto v6.6-rc1 > - add patch to deal with references to PTE_MAYBE_NG from asm code > - add patch to move dummy 'nokaslr' parsing routine out of > idreg-override.c > - rework ptdump address marker array population > > NOTE: this series still does not address the TLBI changes needed for > LPA2 and 5 level paging. Ryan seems to have a good handle on those, and > this work is complementary with his KVM work to a fair extent anyway. As per the above note, I think this series would be broken on a system that supports both LPA2 and TLB_RANGE. The issue is that the BADDR field is specified in 64K units when LPA2 is enabled, but in PAGE_SIZE units when LPA2 is disabled. I think this patch set will continue to set BADDR in PAGE_SIZE units when LPA2 is enabled, causing the HW to invalidate the wrong range? My patch at [1] solves this. I'm currently doing some benchmarking refactoring the patches into a differnet shape as requested by Mark. Anyway, I wonder if this is a blocker for merging this series? [1] https://lore.kernel.org/kvmarm/20231009185008.3803879-3-ryan.roberts@arm.com/ Thanks, Ryan
On Thu, Oct 26, 2023 at 02:21:26PM +0100, Ryan Roberts wrote: > On 12/09/2023 15:15, Ard Biesheuvel wrote: > > From: Ard Biesheuvel <ardb@kernel.org> > > > > This is a followup to [0], which was sent out more than 6 months ago. > > Thanks to Ryan and Mark for feedback and review. This series is > > independent from Ryan's work on adding support for LPA2 to KVM - the > > only potential source of conflict should be the patch "arm64: kvm: Limit > > HYP VA and host S2 range to 48 bits when LPA2 is in effect", which could > > simply be dropped in favour of the KVM changes to make it support LPA2. > > > > Changes since v3: > > - add some acks and incorporate some minor suggested tweaks, mostly > > related to coding style and comments > > - rebase onto v6.6-rc1 > > - add patch to deal with references to PTE_MAYBE_NG from asm code > > - add patch to move dummy 'nokaslr' parsing routine out of > > idreg-override.c > > - rework ptdump address marker array population > > > > NOTE: this series still does not address the TLBI changes needed for > > LPA2 and 5 level paging. Ryan seems to have a good handle on those, and > > this work is complementary with his KVM work to a fair extent anyway. > > As per the above note, I think this series would be broken on a system that > supports both LPA2 and TLB_RANGE. The issue is that the BADDR field is specified > in 64K units when LPA2 is enabled, but in PAGE_SIZE units when LPA2 is disabled. > I think this patch set will continue to set BADDR in PAGE_SIZE units when LPA2 > is enabled, causing the HW to invalidate the wrong range? > > My patch at [1] solves this. I'm currently doing some benchmarking refactoring > the patches into a differnet shape as requested by Mark. > > Anyway, I wonder if this is a blocker for merging this series? > > [1] https://lore.kernel.org/kvmarm/20231009185008.3803879-3-ryan.roberts@arm.com/ Thanks Ryan. I missed this part. While I could add your patch on top, we are debugging some CI reports, so there's a good chance that I'll drop the whole branch later today. I was hoping I can keep part of the series but it probably makes more sense to merge the first 15-20 patches after -rc1 (no new feature added) and keep them in next for a while. We don't have time before the merging window to test what other CIs report (and we haven't managed to reproduce the errors either).
On 26/10/2023 14:52, Catalin Marinas wrote: > On Thu, Oct 26, 2023 at 02:21:26PM +0100, Ryan Roberts wrote: >> On 12/09/2023 15:15, Ard Biesheuvel wrote: >>> From: Ard Biesheuvel <ardb@kernel.org> >>> >>> This is a followup to [0], which was sent out more than 6 months ago. >>> Thanks to Ryan and Mark for feedback and review. This series is >>> independent from Ryan's work on adding support for LPA2 to KVM - the >>> only potential source of conflict should be the patch "arm64: kvm: Limit >>> HYP VA and host S2 range to 48 bits when LPA2 is in effect", which could >>> simply be dropped in favour of the KVM changes to make it support LPA2. >>> >>> Changes since v3: >>> - add some acks and incorporate some minor suggested tweaks, mostly >>> related to coding style and comments >>> - rebase onto v6.6-rc1 >>> - add patch to deal with references to PTE_MAYBE_NG from asm code >>> - add patch to move dummy 'nokaslr' parsing routine out of >>> idreg-override.c >>> - rework ptdump address marker array population >>> >>> NOTE: this series still does not address the TLBI changes needed for >>> LPA2 and 5 level paging. Ryan seems to have a good handle on those, and >>> this work is complementary with his KVM work to a fair extent anyway. >> >> As per the above note, I think this series would be broken on a system that >> supports both LPA2 and TLB_RANGE. The issue is that the BADDR field is specified >> in 64K units when LPA2 is enabled, but in PAGE_SIZE units when LPA2 is disabled. >> I think this patch set will continue to set BADDR in PAGE_SIZE units when LPA2 >> is enabled, causing the HW to invalidate the wrong range? >> >> My patch at [1] solves this. I'm currently doing some benchmarking refactoring >> the patches into a differnet shape as requested by Mark. >> >> Anyway, I wonder if this is a blocker for merging this series? >> >> [1] https://lore.kernel.org/kvmarm/20231009185008.3803879-3-ryan.roberts@arm.com/ > > Thanks Ryan. I missed this part. While I could add your patch on top, we > are debugging some CI reports, so there's a good chance that I'll drop > the whole branch later today. > > I was hoping I can keep part of the series but it probably makes more > sense to merge the first 15-20 patches after -rc1 (no new feature added) > and keep them in next for a while. We don't have time before the merging > window to test what other CIs report (and we haven't managed to > reproduce the errors either). > OK that's a shame. I was hoping this would go in this cycle, then I could add the KVM changes on top and smooth it all out for the cycle after. I guess we will figure out how to land them both at the same time for 6.8.
On Mon, Oct 23, 2023 at 06:45:25PM +0100, Catalin Marinas wrote: > On Tue, 12 Sep 2023 14:15:50 +0000, Ard Biesheuvel wrote: > > This is a followup to [0], which was sent out more than 6 months ago. > > Thanks to Ryan and Mark for feedback and review. This series is > > independent from Ryan's work on adding support for LPA2 to KVM - the > > only potential source of conflict should be the patch "arm64: kvm: Limit > > HYP VA and host S2 range to 48 bits when LPA2 is in effect", which could > > simply be dropped in favour of the KVM changes to make it support LPA2. > > > > [...] > > I pushed the series to the arm64 for-next/lpa2-stage1 branch. If > something falls apart badly in -next (other than the typical conflicts), > I can drop the series before the upcoming merging window. And we got to the point where, unfortunately, I have to drop this series. There's isn't much time before the merging window to investigate the CI reports. I was hoping I can keep the first 15 or so patches but I'm getting some failures with them (which may or may not be related). We'll try again after -rc1 but maybe in smaller pieces: a series reworking the VA range and moving the init to C, a second series for LPA2 stage 1 and a third with Ryan's patches for stage 2. Oliver, please drop this branch from the arm64 kvm tree and sorry for the additional work.
From: Ard Biesheuvel <ardb@kernel.org> This is a followup to [0], which was sent out more than 6 months ago. Thanks to Ryan and Mark for feedback and review. This series is independent from Ryan's work on adding support for LPA2 to KVM - the only potential source of conflict should be the patch "arm64: kvm: Limit HYP VA and host S2 range to 48 bits when LPA2 is in effect", which could simply be dropped in favour of the KVM changes to make it support LPA2. Changes since v3: - add some acks and incorporate some minor suggested tweaks, mostly related to coding style and comments - rebase onto v6.6-rc1 - add patch to deal with references to PTE_MAYBE_NG from asm code - add patch to move dummy 'nokaslr' parsing routine out of idreg-override.c - rework ptdump address marker array population NOTE: this series still does not address the TLBI changes needed for LPA2 and 5 level paging. Ryan seems to have a good handle on those, and this work is complementary with his KVM work to a fair extent anyway. -%- The first ~15 patches of this series rework how the kernel VA space is organized, so that the vmemmap region does not take up more space than necessary, and so that most of it can be reclaimed when running a build capable of 52-bit virtual addressing on hardware that is not. This is needed because the vmemmap region will take up a substantial part of the upper VA region that it shares with the kernel, modules and vmalloc/vmap mappings once we enable LPA2 with 4k pages. The next ~30 patches rework the early init code, reimplementing most of the page table and relocation handling in C code. There are several reasons why this is beneficial: - we generally prefer C code over asm for these things, and the macros that currently exist in head.S for creating the kernel pages tables are a good example why; - we no longer need to create the kernel mapping in two passes, which means we can remove the logic that copies parts of the fixmap and the KAsan shadow from one set of page tables to the other; this is especially advantageous for KAsan with LPA2, which needs more elaborate shadow handling across multiple levels, since the KAsan region cannot be placed on exact pgd_t boundaries in that case; - we can read the ID registers and parse command line overrides before creating the page tables, which simplifies the LPA2 case, as flicking the global TCR_EL1.DS bit at a later stage would require elaborate repainting of all page table descriptors, some of which with the MMU disabled; - we can use more elaborate logic to create the mappings, which means we can use more precise mappings for code and data sections even when using 2 MiB granularity, and this is a prerequisite for running with WXN. As part of the ID map changes, we decouple the ID map size from the kernel VA size, and switch to a 48-bit VA map for all configurations. The next 18 patches rework the existing LVA support as a CPU feature, which simplifies some code and gets rid of the vabits_actual variable. Then, LPA2 support is implemented in the same vein. This requires adding support for 5 level paging as well, given that LPA2 introduces a new paging level '-1' when using 4k pages. Combined with the vmemmap changes at the start of the series, the resulting LPA2/4k pages configuration will have the exact same VA space layout as the ordinary 4k/4 levels configuration, and so LPA2 support can reasonably be enabled by default, as the fallback is seamless on non-LPA2 hardware. In the 16k/LPA2 case, the fallback also reduces the number of paging levels, resulting in a 47-bit VA space. This is based on the assumption that hybrid LPA2/non-LPA2 16k pages kernels in production use would prefer not to take the performance hit of 4 level paging to gain only a single additional bit of VA space. (Note that generic Android kernels use only 3 levels of paging today.) Bespoke 16k configurations can still configure 48-bit virtual addressing as before. Finally, enable support for running with the WXN control enabled. This was previously part of a separate series [1], but given that the delta is tiny, it is included here as well. [0] https://lore.kernel.org/all/20230307140522.2311461-1-ardb@kernel.org/ [1] https://lore.kernel.org/all/20221111171201.2088501-1-ardb@kernel.org/ Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: Marc Zyngier <maz@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Anshuman Khandual <anshuman.khandual@arm.com> Cc: Kees Cook <keescook@chromium.org> Cc: Joey Gouly <joey.gouly@arm.com> Anshuman Khandual (2): arm64/mm: Add FEAT_LPA2 specific TCR_EL1.DS field arm64/mm: Add FEAT_LPA2 specific ID_AA64MMFR0.TGRAN[2] Ard Biesheuvel (59): arm64: kernel: Disable latent_entropy GCC plugin in early C runtime arm64: mm: Take potential load offset into account when KASLR is off arm64: mm: get rid of kimage_vaddr global variable arm64: mm: Move PCI I/O emulation region above the vmemmap region arm64: mm: Move fixmap region above vmemmap region arm64: ptdump: Allow all region boundaries to be defined at boot time arm64: ptdump: Discover start of vmemmap region at runtime arm64: vmemmap: Avoid base2 order of struct page size to dimension region arm64: mm: Reclaim unused vmemmap region for vmalloc use arm64: kaslr: Adjust randomization range dynamically arm64: kernel: Manage absolute relocations in code built under pi/ arm64: kernel: Don't rely on objcopy to make code under pi/ __init arm64: head: move relocation handling to C code arm64: idreg-override: Omit non-NULL checks for override pointer arm64: idreg-override: Prepare for place relative reloc patching arm64: idreg-override: Avoid parameq() and parameqn() arm64: idreg-override: avoid strlen() to check for empty strings arm64: idreg-override: Avoid sprintf() for simple string concatenation arm64: idreg-override: Avoid kstrtou64() to parse a single hex digit arm64: idreg-override: Move to early mini C runtime arm64: kernel: Remove early fdt remap code arm64: head: Clear BSS and the kernel page tables in one go arm64: Move feature overrides into the BSS section arm64: head: Run feature override detection before mapping the kernel arm64: head: move dynamic shadow call stack patching into early C runtime arm64: kaslr: Use feature override instead of parsing the cmdline again arm64/kernel: Move 'nokaslr' parsing out of early idreg code arm64: idreg-override: Create a pseudo feature for rodata=off arm64: Add helpers to probe local CPU for PAC and BTI support arm64: head: allocate more pages for the kernel mapping arm64: head: move memstart_offset_seed handling to C code arm64: mm: Make kaslr_requires_kpti() a static inline arm64: head: Move early kernel mapping routines into C code arm64: mm: Use 48-bit virtual addressing for the permanent ID map arm64: pgtable: Decouple PGDIR size macros from PGD/PUD/PMD levels arm64: kernel: Create initial ID map from C code arm64: mm: avoid fixmap for early swapper_pg_dir updates arm64: mm: omit redundant remap of kernel image arm64: Revert "mm: provide idmap pointer to cpu_replace_ttbr1()" arm64: mmu: Make cpu_replace_ttbr1() out of line arm64: mm: Handle LVA support as a CPU feature arm64: mm: Add feature override support for LVA arm64/mm: Avoid #define'ing PTE_MAYBE_NG to 0x0 for asm use arm64: mm: Wire up TCR.DS bit to PTE shareability fields arm64: mm: Add LPA2 support to phys<->pte conversion routines arm64: mm: Add definitions to support 5 levels of paging arm64: mm: add LPA2 and 5 level paging support to G-to-nG conversion arm64: Enable LPA2 at boot if supported by the system arm64: mm: Add 5 level paging support to fixmap and swapper handling arm64: kasan: Reduce minimum shadow alignment and enable 5 level paging arm64: mm: Add support for folding PUDs at runtime arm64: ptdump: Disregard unaddressable VA space arm64: ptdump: Deal with translation levels folded at runtime arm64: kvm: avoid CONFIG_PGTABLE_LEVELS for runtime levels arm64: kvm: Limit HYP VA and host S2 range to 48 bits when LPA2 is in effect arm64: Enable 52-bit virtual addressing for 4k and 16k granule configs arm64: defconfig: Enable LPA2 support mm: add arch hook to validate mmap() prot flags arm64: mm: add support for WXN memory translation attribute arch/arm64/Kconfig | 34 +- arch/arm64/configs/defconfig | 2 +- arch/arm64/include/asm/archrandom.h | 2 - arch/arm64/include/asm/assembler.h | 55 +-- arch/arm64/include/asm/cpufeature.h | 94 ++++ arch/arm64/include/asm/fixmap.h | 2 +- arch/arm64/include/asm/kasan.h | 2 - arch/arm64/include/asm/kernel-pgtable.h | 128 ++---- arch/arm64/include/asm/memory.h | 37 +- arch/arm64/include/asm/mman.h | 36 ++ arch/arm64/include/asm/mmu.h | 40 +- arch/arm64/include/asm/mmu_context.h | 86 ++-- arch/arm64/include/asm/pgalloc.h | 53 ++- arch/arm64/include/asm/pgtable-hwdef.h | 33 +- arch/arm64/include/asm/pgtable-prot.h | 22 +- arch/arm64/include/asm/pgtable-types.h | 6 + arch/arm64/include/asm/pgtable.h | 229 +++++++++- arch/arm64/include/asm/scs.h | 36 +- arch/arm64/include/asm/setup.h | 3 - arch/arm64/include/asm/sysreg.h | 2 + arch/arm64/include/asm/tlb.h | 3 +- arch/arm64/kernel/Makefile | 7 +- arch/arm64/kernel/cpufeature.c | 104 +++-- arch/arm64/kernel/head.S | 465 ++------------------ arch/arm64/kernel/image-vars.h | 35 ++ arch/arm64/kernel/kaslr.c | 11 +- arch/arm64/kernel/module.c | 2 +- arch/arm64/kernel/pi/Makefile | 28 +- arch/arm64/kernel/{ => pi}/idreg-override.c | 221 ++++++---- arch/arm64/kernel/pi/kaslr_early.c | 78 +--- arch/arm64/kernel/pi/map_kernel.c | 278 ++++++++++++ arch/arm64/kernel/pi/map_range.c | 105 +++++ arch/arm64/kernel/{ => pi}/patch-scs.c | 36 +- arch/arm64/kernel/pi/pi.h | 30 ++ arch/arm64/kernel/pi/relacheck.c | 130 ++++++ arch/arm64/kernel/pi/relocate.c | 64 +++ arch/arm64/kernel/setup.c | 22 - arch/arm64/kernel/sleep.S | 3 - arch/arm64/kernel/suspend.c | 2 +- arch/arm64/kernel/vmlinux.lds.S | 17 +- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 2 + arch/arm64/kvm/mmu.c | 22 +- arch/arm64/kvm/va_layout.c | 9 +- arch/arm64/mm/fixmap.c | 36 +- arch/arm64/mm/init.c | 2 +- arch/arm64/mm/kasan_init.c | 154 +++++-- arch/arm64/mm/mmap.c | 4 + arch/arm64/mm/mmu.c | 267 ++++++----- arch/arm64/mm/pgd.c | 17 +- arch/arm64/mm/proc.S | 122 ++++- arch/arm64/mm/ptdump.c | 77 ++-- arch/arm64/tools/cpucaps | 1 + include/linux/mman.h | 15 + mm/mmap.c | 3 + 54 files changed, 2078 insertions(+), 1196 deletions(-) rename arch/arm64/kernel/{ => pi}/idreg-override.c (53%) create mode 100644 arch/arm64/kernel/pi/map_kernel.c create mode 100644 arch/arm64/kernel/pi/map_range.c rename arch/arm64/kernel/{ => pi}/patch-scs.c (89%) create mode 100644 arch/arm64/kernel/pi/pi.h create mode 100644 arch/arm64/kernel/pi/relacheck.c create mode 100644 arch/arm64/kernel/pi/relocate.c