| Message ID | E1ilAiY-0000MA-RG@rmk-PC.armlinux.org.uk (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v2] arm64: kvm: fix IDMAP overlap with HYP VA | expand |
On Sat, 28 Dec 2019 11:57:14 +0000 Russell King <rmk+kernel@armlinux.org.uk> wrote: > Booting 5.4 on LX2160A reveals that KVM is non-functional: > > kvm: Limiting the IPA size due to kernel Virtual Address limit > kvm [1]: IPA Size Limit: 43bits > kvm [1]: IDMAP intersecting with HYP VA, unable to continue > kvm [1]: error initializing Hyp mode: -22 > > Debugging shows: > > kvm [1]: IDMAP page: 81a26000 > kvm [1]: HYP VA range: 0:22ffffffff > > as RAM is located at: > > 80000000-fbdfffff : System RAM > 2080000000-237fffffff : System RAM > > Comparing this with the same kernel on Armada 8040 shows: > > kvm: Limiting the IPA size due to kernel Virtual Address limit > kvm [1]: IPA Size Limit: 43bits > kvm [1]: IDMAP page: 2a26000 > kvm [1]: HYP VA range: 4800000000:493fffffff > ... > kvm [1]: Hyp mode initialized successfully > > which indicates that hyp_va_msb is set, and is always set to the > opposite value of the idmap page to avoid the overlap. This does not > happen with the LX2160A. > > Further debugging shows vabits_actual = 39, kva_msb = 38 on LX2160A and > kva_msb = 33 on Armada 8040. Looking at the bit layout of the HYP VA, > there is still one bit available for hyp_va_msb. Set this bit > appropriately. This allows kvm to be functional on the LX2160A, but > without any HYP VA randomisation: > > kvm: Limiting the IPA size due to kernel Virtual Address limit > kvm [1]: IPA Size Limit: 43bits > kvm [1]: IDMAP page: 81a24000 > kvm [1]: HYP VA range: 4000000000:62ffffffff > ... > kvm [1]: Hyp mode initialized successfully > > Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk> I've applied this to kvmarm-next with a couple of cleanups, and preserving the fallback when the tag is zero (only the mask gets applied, without any ROR or ADD). M.
On Sun, Jan 19, 2020 at 05:43:27PM +0000, Marc Zyngier wrote: > On Sat, 28 Dec 2019 11:57:14 +0000 > Russell King <rmk+kernel@armlinux.org.uk> wrote: > > > Booting 5.4 on LX2160A reveals that KVM is non-functional: > > > > kvm: Limiting the IPA size due to kernel Virtual Address limit > > kvm [1]: IPA Size Limit: 43bits > > kvm [1]: IDMAP intersecting with HYP VA, unable to continue > > kvm [1]: error initializing Hyp mode: -22 > > > > Debugging shows: > > > > kvm [1]: IDMAP page: 81a26000 > > kvm [1]: HYP VA range: 0:22ffffffff > > > > as RAM is located at: > > > > 80000000-fbdfffff : System RAM > > 2080000000-237fffffff : System RAM > > > > Comparing this with the same kernel on Armada 8040 shows: > > > > kvm: Limiting the IPA size due to kernel Virtual Address limit > > kvm [1]: IPA Size Limit: 43bits > > kvm [1]: IDMAP page: 2a26000 > > kvm [1]: HYP VA range: 4800000000:493fffffff > > ... > > kvm [1]: Hyp mode initialized successfully > > > > which indicates that hyp_va_msb is set, and is always set to the > > opposite value of the idmap page to avoid the overlap. This does not > > happen with the LX2160A. > > > > Further debugging shows vabits_actual = 39, kva_msb = 38 on LX2160A and > > kva_msb = 33 on Armada 8040. Looking at the bit layout of the HYP VA, > > there is still one bit available for hyp_va_msb. Set this bit > > appropriately. This allows kvm to be functional on the LX2160A, but > > without any HYP VA randomisation: > > > > kvm: Limiting the IPA size due to kernel Virtual Address limit > > kvm [1]: IPA Size Limit: 43bits > > kvm [1]: IDMAP page: 81a24000 > > kvm [1]: HYP VA range: 4000000000:62ffffffff > > ... > > kvm [1]: Hyp mode initialized successfully > > > > Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk> > > I've applied this to kvmarm-next with a couple of cleanups, and > preserving the fallback when the tag is zero (only the mask gets > applied, without any ROR or ADD). If only the mask is applied, then it will overlap with the IDMAP region, and KVM will fail - so I think it would be a good idea in that case to print something a little more useful, rather than attributing the KVM failure to an overlap of IDMAP and the KVM range. The real problem is there aren't enough VA bits to allow the KVM range to be adequately placed, rather than the overlap itself.
On Sun, 19 Jan 2020 19:43:40 +0000 Russell King - ARM Linux admin <linux@armlinux.org.uk> wrote: > On Sun, Jan 19, 2020 at 05:43:27PM +0000, Marc Zyngier wrote: > > On Sat, 28 Dec 2019 11:57:14 +0000 > > Russell King <rmk+kernel@armlinux.org.uk> wrote: > > > > > Booting 5.4 on LX2160A reveals that KVM is non-functional: > > > > > > kvm: Limiting the IPA size due to kernel Virtual Address limit > > > kvm [1]: IPA Size Limit: 43bits > > > kvm [1]: IDMAP intersecting with HYP VA, unable to continue > > > kvm [1]: error initializing Hyp mode: -22 > > > > > > Debugging shows: > > > > > > kvm [1]: IDMAP page: 81a26000 > > > kvm [1]: HYP VA range: 0:22ffffffff > > > > > > as RAM is located at: > > > > > > 80000000-fbdfffff : System RAM > > > 2080000000-237fffffff : System RAM > > > > > > Comparing this with the same kernel on Armada 8040 shows: > > > > > > kvm: Limiting the IPA size due to kernel Virtual Address limit > > > kvm [1]: IPA Size Limit: 43bits > > > kvm [1]: IDMAP page: 2a26000 > > > kvm [1]: HYP VA range: 4800000000:493fffffff > > > ... > > > kvm [1]: Hyp mode initialized successfully > > > > > > which indicates that hyp_va_msb is set, and is always set to the > > > opposite value of the idmap page to avoid the overlap. This does not > > > happen with the LX2160A. > > > > > > Further debugging shows vabits_actual = 39, kva_msb = 38 on LX2160A and > > > kva_msb = 33 on Armada 8040. Looking at the bit layout of the HYP VA, > > > there is still one bit available for hyp_va_msb. Set this bit > > > appropriately. This allows kvm to be functional on the LX2160A, but > > > without any HYP VA randomisation: > > > > > > kvm: Limiting the IPA size due to kernel Virtual Address limit > > > kvm [1]: IPA Size Limit: 43bits > > > kvm [1]: IDMAP page: 81a24000 > > > kvm [1]: HYP VA range: 4000000000:62ffffffff > > > ... > > > kvm [1]: Hyp mode initialized successfully > > > > > > Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk> > > > > I've applied this to kvmarm-next with a couple of cleanups, and > > preserving the fallback when the tag is zero (only the mask gets > > applied, without any ROR or ADD). > > If only the mask is applied, then it will overlap with the IDMAP > region, and KVM will fail If the tag (which includes the V-1 bit) is *zero*, what else would you add? > - so I think it would be a good idea in > that case to print something a little more useful, rather than > attributing the KVM failure to an overlap of IDMAP and the KVM > range. What other failure mode do you anticipate? > The real problem is there aren't enough VA bits to allow the KVM > range to be adequately placed, rather than the overlap itself. I don't get your point. By construction, there *are* enough VA bits, since EL2 is only concerned with the linear mapping which only occupies (at most) half of that VA space. If we can't do that at EL2, then we can't do it at EL1 either. M.
diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c index 2cf7d4b606c3..1598909afbaf 100644 --- a/arch/arm64/kvm/va_layout.c +++ b/arch/arm64/kvm/va_layout.c @@ -22,43 +22,37 @@ static u8 tag_lsb; static u64 tag_val; static u64 va_mask; +/* + * We want to generate a hyp VA with the following format (with V == + * vabits_actual): + * + * 63 ... V | V-1 | V-2 .. tag_lsb | tag_lsb - 1 .. 0 + * --------------------------------------------------------- + * | 0000000 | hyp_va_msb | random tag | kern linear VA | + * |--------- tag_val -----------|----- va_mask ---| + * + * which does not conflict with the idmap regions. + */ static void compute_layout(void) { phys_addr_t idmap_addr = __pa_symbol(__hyp_idmap_text_start); u64 hyp_va_msb; - int kva_msb; /* Where is my RAM region? */ hyp_va_msb = idmap_addr & BIT(vabits_actual - 1); hyp_va_msb ^= BIT(vabits_actual - 1); - kva_msb = fls64((u64)phys_to_virt(memblock_start_of_DRAM()) ^ + tag_lsb = fls64((u64)phys_to_virt(memblock_start_of_DRAM()) ^ (u64)(high_memory - 1)); - if (kva_msb == (vabits_actual - 1)) { - /* - * No space in the address, let's compute the mask so - * that it covers (vabits_actual - 1) bits, and the region - * bit. The tag stays set to zero. - */ - va_mask = BIT(vabits_actual - 1) - 1; - va_mask |= hyp_va_msb; - } else { - /* - * We do have some free bits to insert a random tag. - * Hyp VAs are now created from kernel linear map VAs - * using the following formula (with V == vabits_actual): - * - * 63 ... V | V-1 | V-2 .. tag_lsb | tag_lsb - 1 .. 0 - * --------------------------------------------------------- - * | 0000000 | hyp_va_msb | random tag | kern linear VA | - */ - tag_lsb = kva_msb; - va_mask = GENMASK_ULL(tag_lsb - 1, 0); - tag_val = get_random_long() & GENMASK_ULL(vabits_actual - 2, tag_lsb); - tag_val |= hyp_va_msb; - tag_val >>= tag_lsb; + va_mask = GENMASK_ULL(tag_lsb - 1, 0); + tag_val = hyp_va_msb; + + if (tag_lsb != (vabits_actual - 1)) { + /* We have some free bits to insert a random tag. */ + tag_val |= get_random_long() & GENMASK_ULL(vabits_actual - 2, tag_lsb); } + tag_val >>= tag_lsb; } static u32 compute_instruction(int n, u32 rd, u32 rn)
Booting 5.4 on LX2160A reveals that KVM is non-functional: kvm: Limiting the IPA size due to kernel Virtual Address limit kvm [1]: IPA Size Limit: 43bits kvm [1]: IDMAP intersecting with HYP VA, unable to continue kvm [1]: error initializing Hyp mode: -22 Debugging shows: kvm [1]: IDMAP page: 81a26000 kvm [1]: HYP VA range: 0:22ffffffff as RAM is located at: 80000000-fbdfffff : System RAM 2080000000-237fffffff : System RAM Comparing this with the same kernel on Armada 8040 shows: kvm: Limiting the IPA size due to kernel Virtual Address limit kvm [1]: IPA Size Limit: 43bits kvm [1]: IDMAP page: 2a26000 kvm [1]: HYP VA range: 4800000000:493fffffff ... kvm [1]: Hyp mode initialized successfully which indicates that hyp_va_msb is set, and is always set to the opposite value of the idmap page to avoid the overlap. This does not happen with the LX2160A. Further debugging shows vabits_actual = 39, kva_msb = 38 on LX2160A and kva_msb = 33 on Armada 8040. Looking at the bit layout of the HYP VA, there is still one bit available for hyp_va_msb. Set this bit appropriately. This allows kvm to be functional on the LX2160A, but without any HYP VA randomisation: kvm: Limiting the IPA size due to kernel Virtual Address limit kvm [1]: IPA Size Limit: 43bits kvm [1]: IDMAP page: 81a24000 kvm [1]: HYP VA range: 4000000000:62ffffffff ... kvm [1]: Hyp mode initialized successfully Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk> --- arch/arm64/kvm/va_layout.c | 44 ++++++++++++++++---------------------- 1 file changed, 19 insertions(+), 25 deletions(-)