diff mbox series

[v4,1/2] iommu/arm-smmu-v3: Match Stall behaviour for S2

Message ID 20240830110349.797399-2-smostafa@google.com (mailing list archive)
State New, archived
Headers show
Series Fix handling of S2 stalls | expand

Commit Message

Mostafa Saleh Aug. 30, 2024, 11:03 a.m. UTC
According to the spec (ARM IHI 0070 F.b), in
"5.5 Fault configuration (A, R, S bits)":
    A STE with stage 2 translation enabled and STE.S2S == 0 is
    considered ILLEGAL if SMMU_IDR0.STALL_MODEL == 0b10.

Also described in the pseudocode “SteIllegal()”
    if STE.Config == '11x' then
        [..]
        if eff_idr0_stall_model == '10' && STE.S2S == '0' then
            // stall_model forcing stall, but S2S == 0
            return TRUE;

Which means, S2S must be set when stall model is
"ARM_SMMU_FEAT_STALL_FORCE", but currently the driver ignores that.

Although, the driver can do the minimum and only set S2S for
“ARM_SMMU_FEAT_STALL_FORCE”, it is more consistent to match S1
behaviour, which also sets it for “ARM_SMMU_FEAT_STALL” if the
master has requested stalls.

Also, since S2 stalls are enabled now, report them to the IOMMU layer
and for VFIO devices it will fail anyway as VFIO doesn’t register an
iopf handler.

Signed-off-by: Mostafa Saleh <smostafa@google.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 8 +++-----
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 +
 2 files changed, 4 insertions(+), 5 deletions(-)

Comments

Nicolin Chen Aug. 30, 2024, 4:30 p.m. UTC | #1
On Fri, Aug 30, 2024 at 11:03:47AM +0000, Mostafa Saleh wrote:
 
> According to the spec (ARM IHI 0070 F.b), in
> "5.5 Fault configuration (A, R, S bits)":
>     A STE with stage 2 translation enabled and STE.S2S == 0 is
>     considered ILLEGAL if SMMU_IDR0.STALL_MODEL == 0b10.
> 
> Also described in the pseudocode “SteIllegal()”
>     if STE.Config == '11x' then
>         [..]
>         if eff_idr0_stall_model == '10' && STE.S2S == '0' then
>             // stall_model forcing stall, but S2S == 0
>             return TRUE;
> 
> Which means, S2S must be set when stall model is
> "ARM_SMMU_FEAT_STALL_FORCE", but currently the driver ignores that.
> 
> Although, the driver can do the minimum and only set S2S for
> “ARM_SMMU_FEAT_STALL_FORCE”, it is more consistent to match S1
> behaviour, which also sets it for “ARM_SMMU_FEAT_STALL” if the
> master has requested stalls.

If I read the SteIllegal() correctly, it seems S2S would conflict
against the STE.EATS settings?

// Check ATS configuration
if ((sec_sid == SS_NonSecure && SMMU_IDR0.ATS == '1') ||
    (sec_sid == SS_Realm && SMMU_R_IDR0.ATS == '1')) &&
    STE.Config != 'x00' then
    // Needs to be NS/Realm, ATS enabled, and not Bypass
        if STE.EATS == '01' && STE.S2S == '1' then
            // Full ATS mode
            if STE.Config == '11x' || constr_unpred_EATS_S2S then
                // if stage 2 enabled or CONSTRAINED UNPREDICTABLE for SMMUv3.0
                return TRUE;
	
So, if master->stall_enabled and master->ats_enabled, there would
be a bad STE?

Thanks
Nicolin
Robin Murphy Aug. 30, 2024, 5:02 p.m. UTC | #2
On 30/08/2024 5:30 pm, Nicolin Chen wrote:
> On Fri, Aug 30, 2024 at 11:03:47AM +0000, Mostafa Saleh wrote:
>   
>> According to the spec (ARM IHI 0070 F.b), in
>> "5.5 Fault configuration (A, R, S bits)":
>>      A STE with stage 2 translation enabled and STE.S2S == 0 is
>>      considered ILLEGAL if SMMU_IDR0.STALL_MODEL == 0b10.
>>
>> Also described in the pseudocode “SteIllegal()”
>>      if STE.Config == '11x' then
>>          [..]
>>          if eff_idr0_stall_model == '10' && STE.S2S == '0' then
>>              // stall_model forcing stall, but S2S == 0
>>              return TRUE;
>>
>> Which means, S2S must be set when stall model is
>> "ARM_SMMU_FEAT_STALL_FORCE", but currently the driver ignores that.
>>
>> Although, the driver can do the minimum and only set S2S for
>> “ARM_SMMU_FEAT_STALL_FORCE”, it is more consistent to match S1
>> behaviour, which also sets it for “ARM_SMMU_FEAT_STALL” if the
>> master has requested stalls.
> 
> If I read the SteIllegal() correctly, it seems S2S would conflict
> against the STE.EATS settings?
> 
> // Check ATS configuration
> if ((sec_sid == SS_NonSecure && SMMU_IDR0.ATS == '1') ||
>      (sec_sid == SS_Realm && SMMU_R_IDR0.ATS == '1')) &&
>      STE.Config != 'x00' then
>      // Needs to be NS/Realm, ATS enabled, and not Bypass
>          if STE.EATS == '01' && STE.S2S == '1' then
>              // Full ATS mode
>              if STE.Config == '11x' || constr_unpred_EATS_S2S then
>                  // if stage 2 enabled or CONSTRAINED UNPREDICTABLE for SMMUv3.0
>                  return TRUE;
> 	
> So, if master->stall_enabled and master->ats_enabled, there would
> be a bad STE?

Indeed, but as discussed previously, to get there would require either 
firmware or hardware to bogusly advertise both stall and ATS 
capabilities for the same device, which we decided is beyond the scope 
of what's worth trying to reason about. If a nonsensical system leads to 
obviously blowing up with C_BAD_STE, that's arguably not such a bad thing.

Thanks,
Robin.
Nicolin Chen Aug. 30, 2024, 5:07 p.m. UTC | #3
On Fri, Aug 30, 2024 at 06:02:35PM +0100, Robin Murphy wrote:

> On 30/08/2024 5:30 pm, Nicolin Chen wrote:
> > On Fri, Aug 30, 2024 at 11:03:47AM +0000, Mostafa Saleh wrote:
> > 
> > > According to the spec (ARM IHI 0070 F.b), in
> > > "5.5 Fault configuration (A, R, S bits)":
> > >      A STE with stage 2 translation enabled and STE.S2S == 0 is
> > >      considered ILLEGAL if SMMU_IDR0.STALL_MODEL == 0b10.
> > > 
> > > Also described in the pseudocode “SteIllegal()”
> > >      if STE.Config == '11x' then
> > >          [..]
> > >          if eff_idr0_stall_model == '10' && STE.S2S == '0' then
> > >              // stall_model forcing stall, but S2S == 0
> > >              return TRUE;
> > > 
> > > Which means, S2S must be set when stall model is
> > > "ARM_SMMU_FEAT_STALL_FORCE", but currently the driver ignores that.
> > > 
> > > Although, the driver can do the minimum and only set S2S for
> > > “ARM_SMMU_FEAT_STALL_FORCE”, it is more consistent to match S1
> > > behaviour, which also sets it for “ARM_SMMU_FEAT_STALL” if the
> > > master has requested stalls.
> > 
> > If I read the SteIllegal() correctly, it seems S2S would conflict
> > against the STE.EATS settings?
> > 
> > // Check ATS configuration
> > if ((sec_sid == SS_NonSecure && SMMU_IDR0.ATS == '1') ||
> >      (sec_sid == SS_Realm && SMMU_R_IDR0.ATS == '1')) &&
> >      STE.Config != 'x00' then
> >      // Needs to be NS/Realm, ATS enabled, and not Bypass
> >          if STE.EATS == '01' && STE.S2S == '1' then
> >              // Full ATS mode
> >              if STE.Config == '11x' || constr_unpred_EATS_S2S then
> >                  // if stage 2 enabled or CONSTRAINED UNPREDICTABLE for SMMUv3.0
> >                  return TRUE;
> > 
> > So, if master->stall_enabled and master->ats_enabled, there would
> > be a bad STE?
> 
> Indeed, but as discussed previously, to get there would require either
> firmware or hardware to bogusly advertise both stall and ATS
> capabilities for the same device, which we decided is beyond the scope
> of what's worth trying to reason about. If a nonsensical system leads to
> obviously blowing up with C_BAD_STE, that's arguably not such a bad thing.

Oh, I see. Thanks for the note!

Nicolin
Jason Gunthorpe Aug. 30, 2024, 8:20 p.m. UTC | #4
On Fri, Aug 30, 2024 at 11:03:47AM +0000, Mostafa Saleh wrote:
> According to the spec (ARM IHI 0070 F.b), in
> "5.5 Fault configuration (A, R, S bits)":
>     A STE with stage 2 translation enabled and STE.S2S == 0 is
>     considered ILLEGAL if SMMU_IDR0.STALL_MODEL == 0b10.
> 
> Also described in the pseudocode “SteIllegal()”
>     if STE.Config == '11x' then
>         [..]
>         if eff_idr0_stall_model == '10' && STE.S2S == '0' then
>             // stall_model forcing stall, but S2S == 0
>             return TRUE;
> 
> Which means, S2S must be set when stall model is
> "ARM_SMMU_FEAT_STALL_FORCE", but currently the driver ignores that.
> 
> Although, the driver can do the minimum and only set S2S for
> “ARM_SMMU_FEAT_STALL_FORCE”, it is more consistent to match S1
> behaviour, which also sets it for “ARM_SMMU_FEAT_STALL” if the
> master has requested stalls.

Hum, that is looking a bit out of date
perhaps. ARM_SMMU_FEAT_STALL_FORCE should definately set stall, but
for stall-optional it should probably only be set if a faulting type
domain is installed (probably on a PASID)..

Still looks Ok

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>

Jason
diff mbox series

Patch

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index a31460f9f3d4..a0044ff2facf 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1012,7 +1012,8 @@  void arm_smmu_get_ste_used(const __le64 *ent, __le64 *used_bits)
 		used_bits[2] |=
 			cpu_to_le64(STRTAB_STE_2_S2VMID | STRTAB_STE_2_VTCR |
 				    STRTAB_STE_2_S2AA64 | STRTAB_STE_2_S2ENDI |
-				    STRTAB_STE_2_S2PTW | STRTAB_STE_2_S2R);
+				    STRTAB_STE_2_S2PTW | STRTAB_STE_2_S2S |
+				    STRTAB_STE_2_S2R);
 		used_bits[3] |= cpu_to_le64(STRTAB_STE_3_S2TTB_MASK);
 	}
 
@@ -1646,6 +1647,7 @@  void arm_smmu_make_s2_domain_ste(struct arm_smmu_ste *target,
 		STRTAB_STE_2_S2ENDI |
 #endif
 		STRTAB_STE_2_S2PTW |
+		(master->stall_enabled ? STRTAB_STE_2_S2S : 0) |
 		STRTAB_STE_2_S2R);
 
 	target->data[3] = cpu_to_le64(pgtbl_cfg->arm_lpae_s2_cfg.vttbr &
@@ -1739,10 +1741,6 @@  static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt)
 		return -EOPNOTSUPP;
 	}
 
-	/* Stage-2 is always pinned at the moment */
-	if (evt[1] & EVTQ_1_S2)
-		return -EFAULT;
-
 	if (!(evt[1] & EVTQ_1_STALL))
 		return -EOPNOTSUPP;
 
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 14bca41a981b..0dc7ad43c64c 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -267,6 +267,7 @@  struct arm_smmu_ste {
 #define STRTAB_STE_2_S2AA64		(1UL << 51)
 #define STRTAB_STE_2_S2ENDI		(1UL << 52)
 #define STRTAB_STE_2_S2PTW		(1UL << 54)
+#define STRTAB_STE_2_S2S		(1UL << 57)
 #define STRTAB_STE_2_S2R		(1UL << 58)
 
 #define STRTAB_STE_3_S2TTB_MASK		GENMASK_ULL(51, 4)