| Message ID | 20221107215934.1895478-1-oliver.upton@linux.dev (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | KVM: arm64: Parallel stage-2 fault handling | expand |
On Mon, Nov 7, 2022 at 1:59 PM Oliver Upton <oliver.upton@linux.dev> wrote: > > Convert stage2_map_walker_try_leaf() to use the new break-before-make > helpers, thereby making the handler parallel-aware. As before, avoid the > break-before-make if recreating the existing mapping. Additionally, > retry execution if another vCPU thread is modifying the same PTE. > > Signed-off-by: Oliver Upton <oliver.upton@linux.dev> Reviewed-by: Ben Gardon <bgardon@google.com> > --- > arch/arm64/kvm/hyp/pgtable.c | 26 ++++++++++++-------------- > 1 file changed, 12 insertions(+), 14 deletions(-) > > diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c > index b9f0d792b8d9..238f29389617 100644 > --- a/arch/arm64/kvm/hyp/pgtable.c > +++ b/arch/arm64/kvm/hyp/pgtable.c > @@ -804,18 +804,17 @@ static int stage2_map_walker_try_leaf(const struct kvm_pgtable_visit_ctx *ctx, > else > new = kvm_init_invalid_leaf_owner(data->owner_id); > > - if (stage2_pte_is_counted(ctx->old)) { > - /* > - * Skip updating the PTE if we are trying to recreate the exact > - * same mapping or only change the access permissions. Instead, > - * the vCPU will exit one more time from guest if still needed > - * and then go through the path of relaxing permissions. > - */ > - if (!stage2_pte_needs_update(ctx->old, new)) > - return -EAGAIN; > + /* > + * Skip updating the PTE if we are trying to recreate the exact > + * same mapping or only change the access permissions. Instead, > + * the vCPU will exit one more time from guest if still needed > + * and then go through the path of relaxing permissions. > + */ > + if (!stage2_pte_needs_update(ctx->old, new)) > + return -EAGAIN; > > - stage2_put_pte(ctx, data->mmu, mm_ops); > - } > + if (!stage2_try_break_pte(ctx, data->mmu)) > + return -EAGAIN; > > /* Perform CMOs before installation of the guest stage-2 PTE */ > if (mm_ops->dcache_clean_inval_poc && stage2_pte_cacheable(pgt, new)) > @@ -825,9 +824,8 @@ static int stage2_map_walker_try_leaf(const struct kvm_pgtable_visit_ctx *ctx, > if (mm_ops->icache_inval_pou && stage2_pte_executable(new)) > mm_ops->icache_inval_pou(kvm_pte_follow(new, mm_ops), granule); > > - smp_store_release(ctx->ptep, new); > - if (stage2_pte_is_counted(new)) > - mm_ops->get_page(ctx->ptep); > + stage2_make_pte(ctx, new); > + > if (kvm_phys_is_valid(phys)) > data->phys += granule; > return 0; > -- > 2.38.1.431.g37b22c650d-goog >
diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index b9f0d792b8d9..238f29389617 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -804,18 +804,17 @@ static int stage2_map_walker_try_leaf(const struct kvm_pgtable_visit_ctx *ctx, else new = kvm_init_invalid_leaf_owner(data->owner_id); - if (stage2_pte_is_counted(ctx->old)) { - /* - * Skip updating the PTE if we are trying to recreate the exact - * same mapping or only change the access permissions. Instead, - * the vCPU will exit one more time from guest if still needed - * and then go through the path of relaxing permissions. - */ - if (!stage2_pte_needs_update(ctx->old, new)) - return -EAGAIN; + /* + * Skip updating the PTE if we are trying to recreate the exact + * same mapping or only change the access permissions. Instead, + * the vCPU will exit one more time from guest if still needed + * and then go through the path of relaxing permissions. + */ + if (!stage2_pte_needs_update(ctx->old, new)) + return -EAGAIN; - stage2_put_pte(ctx, data->mmu, mm_ops); - } + if (!stage2_try_break_pte(ctx, data->mmu)) + return -EAGAIN; /* Perform CMOs before installation of the guest stage-2 PTE */ if (mm_ops->dcache_clean_inval_poc && stage2_pte_cacheable(pgt, new)) @@ -825,9 +824,8 @@ static int stage2_map_walker_try_leaf(const struct kvm_pgtable_visit_ctx *ctx, if (mm_ops->icache_inval_pou && stage2_pte_executable(new)) mm_ops->icache_inval_pou(kvm_pte_follow(new, mm_ops), granule); - smp_store_release(ctx->ptep, new); - if (stage2_pte_is_counted(new)) - mm_ops->get_page(ctx->ptep); + stage2_make_pte(ctx, new); + if (kvm_phys_is_valid(phys)) data->phys += granule; return 0;
Convert stage2_map_walker_try_leaf() to use the new break-before-make helpers, thereby making the handler parallel-aware. As before, avoid the break-before-make if recreating the existing mapping. Additionally, retry execution if another vCPU thread is modifying the same PTE. Signed-off-by: Oliver Upton <oliver.upton@linux.dev> --- arch/arm64/kvm/hyp/pgtable.c | 26 ++++++++++++-------------- 1 file changed, 12 insertions(+), 14 deletions(-)