| Message ID | 20240830110349.797399-2-smostafa@google.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | Fix handling of S2 stalls | expand |
On Fri, Aug 30, 2024 at 11:03:47AM +0000, Mostafa Saleh wrote: > According to the spec (ARM IHI 0070 F.b), in > "5.5 Fault configuration (A, R, S bits)": > A STE with stage 2 translation enabled and STE.S2S == 0 is > considered ILLEGAL if SMMU_IDR0.STALL_MODEL == 0b10. > > Also described in the pseudocode “SteIllegal()” > if STE.Config == '11x' then > [..] > if eff_idr0_stall_model == '10' && STE.S2S == '0' then > // stall_model forcing stall, but S2S == 0 > return TRUE; > > Which means, S2S must be set when stall model is > "ARM_SMMU_FEAT_STALL_FORCE", but currently the driver ignores that. > > Although, the driver can do the minimum and only set S2S for > “ARM_SMMU_FEAT_STALL_FORCE”, it is more consistent to match S1 > behaviour, which also sets it for “ARM_SMMU_FEAT_STALL” if the > master has requested stalls. If I read the SteIllegal() correctly, it seems S2S would conflict against the STE.EATS settings? // Check ATS configuration if ((sec_sid == SS_NonSecure && SMMU_IDR0.ATS == '1') || (sec_sid == SS_Realm && SMMU_R_IDR0.ATS == '1')) && STE.Config != 'x00' then // Needs to be NS/Realm, ATS enabled, and not Bypass if STE.EATS == '01' && STE.S2S == '1' then // Full ATS mode if STE.Config == '11x' || constr_unpred_EATS_S2S then // if stage 2 enabled or CONSTRAINED UNPREDICTABLE for SMMUv3.0 return TRUE; So, if master->stall_enabled and master->ats_enabled, there would be a bad STE? Thanks Nicolin
On 30/08/2024 5:30 pm, Nicolin Chen wrote: > On Fri, Aug 30, 2024 at 11:03:47AM +0000, Mostafa Saleh wrote: > >> According to the spec (ARM IHI 0070 F.b), in >> "5.5 Fault configuration (A, R, S bits)": >> A STE with stage 2 translation enabled and STE.S2S == 0 is >> considered ILLEGAL if SMMU_IDR0.STALL_MODEL == 0b10. >> >> Also described in the pseudocode “SteIllegal()” >> if STE.Config == '11x' then >> [..] >> if eff_idr0_stall_model == '10' && STE.S2S == '0' then >> // stall_model forcing stall, but S2S == 0 >> return TRUE; >> >> Which means, S2S must be set when stall model is >> "ARM_SMMU_FEAT_STALL_FORCE", but currently the driver ignores that. >> >> Although, the driver can do the minimum and only set S2S for >> “ARM_SMMU_FEAT_STALL_FORCE”, it is more consistent to match S1 >> behaviour, which also sets it for “ARM_SMMU_FEAT_STALL” if the >> master has requested stalls. > > If I read the SteIllegal() correctly, it seems S2S would conflict > against the STE.EATS settings? > > // Check ATS configuration > if ((sec_sid == SS_NonSecure && SMMU_IDR0.ATS == '1') || > (sec_sid == SS_Realm && SMMU_R_IDR0.ATS == '1')) && > STE.Config != 'x00' then > // Needs to be NS/Realm, ATS enabled, and not Bypass > if STE.EATS == '01' && STE.S2S == '1' then > // Full ATS mode > if STE.Config == '11x' || constr_unpred_EATS_S2S then > // if stage 2 enabled or CONSTRAINED UNPREDICTABLE for SMMUv3.0 > return TRUE; > > So, if master->stall_enabled and master->ats_enabled, there would > be a bad STE? Indeed, but as discussed previously, to get there would require either firmware or hardware to bogusly advertise both stall and ATS capabilities for the same device, which we decided is beyond the scope of what's worth trying to reason about. If a nonsensical system leads to obviously blowing up with C_BAD_STE, that's arguably not such a bad thing. Thanks, Robin.
On Fri, Aug 30, 2024 at 06:02:35PM +0100, Robin Murphy wrote: > On 30/08/2024 5:30 pm, Nicolin Chen wrote: > > On Fri, Aug 30, 2024 at 11:03:47AM +0000, Mostafa Saleh wrote: > > > > > According to the spec (ARM IHI 0070 F.b), in > > > "5.5 Fault configuration (A, R, S bits)": > > > A STE with stage 2 translation enabled and STE.S2S == 0 is > > > considered ILLEGAL if SMMU_IDR0.STALL_MODEL == 0b10. > > > > > > Also described in the pseudocode “SteIllegal()” > > > if STE.Config == '11x' then > > > [..] > > > if eff_idr0_stall_model == '10' && STE.S2S == '0' then > > > // stall_model forcing stall, but S2S == 0 > > > return TRUE; > > > > > > Which means, S2S must be set when stall model is > > > "ARM_SMMU_FEAT_STALL_FORCE", but currently the driver ignores that. > > > > > > Although, the driver can do the minimum and only set S2S for > > > “ARM_SMMU_FEAT_STALL_FORCE”, it is more consistent to match S1 > > > behaviour, which also sets it for “ARM_SMMU_FEAT_STALL” if the > > > master has requested stalls. > > > > If I read the SteIllegal() correctly, it seems S2S would conflict > > against the STE.EATS settings? > > > > // Check ATS configuration > > if ((sec_sid == SS_NonSecure && SMMU_IDR0.ATS == '1') || > > (sec_sid == SS_Realm && SMMU_R_IDR0.ATS == '1')) && > > STE.Config != 'x00' then > > // Needs to be NS/Realm, ATS enabled, and not Bypass > > if STE.EATS == '01' && STE.S2S == '1' then > > // Full ATS mode > > if STE.Config == '11x' || constr_unpred_EATS_S2S then > > // if stage 2 enabled or CONSTRAINED UNPREDICTABLE for SMMUv3.0 > > return TRUE; > > > > So, if master->stall_enabled and master->ats_enabled, there would > > be a bad STE? > > Indeed, but as discussed previously, to get there would require either > firmware or hardware to bogusly advertise both stall and ATS > capabilities for the same device, which we decided is beyond the scope > of what's worth trying to reason about. If a nonsensical system leads to > obviously blowing up with C_BAD_STE, that's arguably not such a bad thing. Oh, I see. Thanks for the note! Nicolin
On Fri, Aug 30, 2024 at 11:03:47AM +0000, Mostafa Saleh wrote: > According to the spec (ARM IHI 0070 F.b), in > "5.5 Fault configuration (A, R, S bits)": > A STE with stage 2 translation enabled and STE.S2S == 0 is > considered ILLEGAL if SMMU_IDR0.STALL_MODEL == 0b10. > > Also described in the pseudocode “SteIllegal()” > if STE.Config == '11x' then > [..] > if eff_idr0_stall_model == '10' && STE.S2S == '0' then > // stall_model forcing stall, but S2S == 0 > return TRUE; > > Which means, S2S must be set when stall model is > "ARM_SMMU_FEAT_STALL_FORCE", but currently the driver ignores that. > > Although, the driver can do the minimum and only set S2S for > “ARM_SMMU_FEAT_STALL_FORCE”, it is more consistent to match S1 > behaviour, which also sets it for “ARM_SMMU_FEAT_STALL” if the > master has requested stalls. Hum, that is looking a bit out of date perhaps. ARM_SMMU_FEAT_STALL_FORCE should definately set stall, but for stall-optional it should probably only be set if a faulting type domain is installed (probably on a PASID).. Still looks Ok Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Jason
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index a31460f9f3d4..a0044ff2facf 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1012,7 +1012,8 @@ void arm_smmu_get_ste_used(const __le64 *ent, __le64 *used_bits) used_bits[2] |= cpu_to_le64(STRTAB_STE_2_S2VMID | STRTAB_STE_2_VTCR | STRTAB_STE_2_S2AA64 | STRTAB_STE_2_S2ENDI | - STRTAB_STE_2_S2PTW | STRTAB_STE_2_S2R); + STRTAB_STE_2_S2PTW | STRTAB_STE_2_S2S | + STRTAB_STE_2_S2R); used_bits[3] |= cpu_to_le64(STRTAB_STE_3_S2TTB_MASK); } @@ -1646,6 +1647,7 @@ void arm_smmu_make_s2_domain_ste(struct arm_smmu_ste *target, STRTAB_STE_2_S2ENDI | #endif STRTAB_STE_2_S2PTW | + (master->stall_enabled ? STRTAB_STE_2_S2S : 0) | STRTAB_STE_2_S2R); target->data[3] = cpu_to_le64(pgtbl_cfg->arm_lpae_s2_cfg.vttbr & @@ -1739,10 +1741,6 @@ static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt) return -EOPNOTSUPP; } - /* Stage-2 is always pinned at the moment */ - if (evt[1] & EVTQ_1_S2) - return -EFAULT; - if (!(evt[1] & EVTQ_1_STALL)) return -EOPNOTSUPP; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 14bca41a981b..0dc7ad43c64c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -267,6 +267,7 @@ struct arm_smmu_ste { #define STRTAB_STE_2_S2AA64 (1UL << 51) #define STRTAB_STE_2_S2ENDI (1UL << 52) #define STRTAB_STE_2_S2PTW (1UL << 54) +#define STRTAB_STE_2_S2S (1UL << 57) #define STRTAB_STE_2_S2R (1UL << 58) #define STRTAB_STE_3_S2TTB_MASK GENMASK_ULL(51, 4)
According to the spec (ARM IHI 0070 F.b), in "5.5 Fault configuration (A, R, S bits)": A STE with stage 2 translation enabled and STE.S2S == 0 is considered ILLEGAL if SMMU_IDR0.STALL_MODEL == 0b10. Also described in the pseudocode “SteIllegal()” if STE.Config == '11x' then [..] if eff_idr0_stall_model == '10' && STE.S2S == '0' then // stall_model forcing stall, but S2S == 0 return TRUE; Which means, S2S must be set when stall model is "ARM_SMMU_FEAT_STALL_FORCE", but currently the driver ignores that. Although, the driver can do the minimum and only set S2S for “ARM_SMMU_FEAT_STALL_FORCE”, it is more consistent to match S1 behaviour, which also sets it for “ARM_SMMU_FEAT_STALL” if the master has requested stalls. Also, since S2 stalls are enabled now, report them to the IOMMU layer and for VFIO devices it will fail anyway as VFIO doesn’t register an iopf handler. Signed-off-by: Mostafa Saleh <smostafa@google.com> --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 8 +++----- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 4 insertions(+), 5 deletions(-)