| Message ID | 20230801063125.34995-1-baolu.lu@linux.intel.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | iommu: Make pasid array per device | expand |
On Tue, Aug 01, 2023 at 02:31:23PM +0800, Lu Baolu wrote: > The PCI PASID enabling interface guarantees that the address space used > by each PASID is unique. This is achieved by checking that the PCI ACS > path is enabled for the device. If the path is not enabled, then the > PASID feature cannot be used. > > if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF)) > return -EINVAL; > > The PASID array is not an attribute of the IOMMU group. It is more > natural to store the PASID array in the per-device IOMMU data. This > makes the code clearer and easier to understand. No functional changes > are intended. Is there a reason to do this? *PCI* requires the ACS/etc because PCI kind of messed up how switches handled PASID so PASID doesn't work otherwise. But there is nothing that says other bus type can't have working (non-PCI) PASID and still have device isolation issues. So unless there is a really strong reason to do this we should keep the PASID list in the group just like the domain. Jason
> From: Jason Gunthorpe <jgg@ziepe.ca> > Sent: Wednesday, August 2, 2023 10:16 PM > > On Tue, Aug 01, 2023 at 02:31:23PM +0800, Lu Baolu wrote: > > The PCI PASID enabling interface guarantees that the address space used > > by each PASID is unique. This is achieved by checking that the PCI ACS > > path is enabled for the device. If the path is not enabled, then the > > PASID feature cannot be used. > > > > if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF)) > > return -EINVAL; > > > > The PASID array is not an attribute of the IOMMU group. It is more > > natural to store the PASID array in the per-device IOMMU data. This > > makes the code clearer and easier to understand. No functional changes > > are intended. > > Is there a reason to do this? > > *PCI* requires the ACS/etc because PCI kind of messed up how switches > handled PASID so PASID doesn't work otherwise. > > But there is nothing that says other bus type can't have working > (non-PCI) PASID and still have device isolation issues. > > So unless there is a really strong reason to do this we should keep > the PASID list in the group just like the domain. > this comes from the consensus in [1]. [1] https://lore.kernel.org/linux-iommu/ZAcyEzN4102gPsWC@nvidia.com/
On Thu, Aug 03, 2023 at 12:44:03AM +0000, Tian, Kevin wrote: > > From: Jason Gunthorpe <jgg@ziepe.ca> > > Sent: Wednesday, August 2, 2023 10:16 PM > > > > On Tue, Aug 01, 2023 at 02:31:23PM +0800, Lu Baolu wrote: > > > The PCI PASID enabling interface guarantees that the address space used > > > by each PASID is unique. This is achieved by checking that the PCI ACS > > > path is enabled for the device. If the path is not enabled, then the > > > PASID feature cannot be used. > > > > > > if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF)) > > > return -EINVAL; > > > > > > The PASID array is not an attribute of the IOMMU group. It is more > > > natural to store the PASID array in the per-device IOMMU data. This > > > makes the code clearer and easier to understand. No functional changes > > > are intended. > > > > Is there a reason to do this? > > > > *PCI* requires the ACS/etc because PCI kind of messed up how switches > > handled PASID so PASID doesn't work otherwise. > > > > But there is nothing that says other bus type can't have working > > (non-PCI) PASID and still have device isolation issues. > > > > So unless there is a really strong reason to do this we should keep > > the PASID list in the group just like the domain. > > > > this comes from the consensus in [1]. > > [1] https://lore.kernel.org/linux-iommu/ZAcyEzN4102gPsWC@nvidia.com/ That consensus was that we don't have PASID support if there is multi-device groups, at least in iommufd.. That makes sense. If we want to change the core code to enforce this that also makes sense But this series is just moving the array? Jason
> From: Jason Gunthorpe <jgg@ziepe.ca> > Sent: Thursday, August 3, 2023 11:19 PM > > On Thu, Aug 03, 2023 at 12:44:03AM +0000, Tian, Kevin wrote: > > > From: Jason Gunthorpe <jgg@ziepe.ca> > > > Sent: Wednesday, August 2, 2023 10:16 PM > > > > > > On Tue, Aug 01, 2023 at 02:31:23PM +0800, Lu Baolu wrote: > > > > The PCI PASID enabling interface guarantees that the address space > used > > > > by each PASID is unique. This is achieved by checking that the PCI ACS > > > > path is enabled for the device. If the path is not enabled, then the > > > > PASID feature cannot be used. > > > > > > > > if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF)) > > > > return -EINVAL; > > > > > > > > The PASID array is not an attribute of the IOMMU group. It is more > > > > natural to store the PASID array in the per-device IOMMU data. This > > > > makes the code clearer and easier to understand. No functional > changes > > > > are intended. > > > > > > Is there a reason to do this? > > > > > > *PCI* requires the ACS/etc because PCI kind of messed up how switches > > > handled PASID so PASID doesn't work otherwise. > > > > > > But there is nothing that says other bus type can't have working > > > (non-PCI) PASID and still have device isolation issues. > > > > > > So unless there is a really strong reason to do this we should keep > > > the PASID list in the group just like the domain. > > > > > > > this comes from the consensus in [1]. > > > > [1] https://lore.kernel.org/linux-iommu/ZAcyEzN4102gPsWC@nvidia.com/ > > That consensus was that we don't have PASID support if there is > multi-device groups, at least in iommufd.. That makes sense. If we > want to change the core code to enforce this that also makes sense > > But this series is just moving the array? > This is a preparation series for supporting PASID in iommufd (will be sent out probably after next version of the nesting series). It only moves the array by taking only PCI into consideration. We could add an explicit enforcement in iommu core for all types of devices.
On 2023/8/3 23:18, Jason Gunthorpe wrote: > On Thu, Aug 03, 2023 at 12:44:03AM +0000, Tian, Kevin wrote: >>> From: Jason Gunthorpe<jgg@ziepe.ca> >>> Sent: Wednesday, August 2, 2023 10:16 PM >>> >>> On Tue, Aug 01, 2023 at 02:31:23PM +0800, Lu Baolu wrote: >>>> The PCI PASID enabling interface guarantees that the address space used >>>> by each PASID is unique. This is achieved by checking that the PCI ACS >>>> path is enabled for the device. If the path is not enabled, then the >>>> PASID feature cannot be used. >>>> >>>> if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF)) >>>> return -EINVAL; >>>> >>>> The PASID array is not an attribute of the IOMMU group. It is more >>>> natural to store the PASID array in the per-device IOMMU data. This >>>> makes the code clearer and easier to understand. No functional changes >>>> are intended. >>> Is there a reason to do this? >>> >>> *PCI* requires the ACS/etc because PCI kind of messed up how switches >>> handled PASID so PASID doesn't work otherwise. >>> >>> But there is nothing that says other bus type can't have working >>> (non-PCI) PASID and still have device isolation issues. >>> >>> So unless there is a really strong reason to do this we should keep >>> the PASID list in the group just like the domain. >>> >> this comes from the consensus in [1]. >> >> [1]https://lore.kernel.org/linux-iommu/ZAcyEzN4102gPsWC@nvidia.com/ > That consensus was that we don't have PASID support if there is > multi-device groups, at least in iommufd.. That makes sense. If we > want to change the core code to enforce this that also makes sense In my initial plan, I had a third patch that would have enforced single- device groups for PASID interfaces in the core. But I ultimately dropped it because it is the fact for PCI devices, but I am not sure about other buses although perhaps there is none. > But this series is just moving the array? So I took the first step by moving the pasid_array from iommu group to the device. :-) Best regards, baolu
On 2023/8/4 10:20, Baolu Lu wrote: > On 2023/8/3 23:18, Jason Gunthorpe wrote: >> On Thu, Aug 03, 2023 at 12:44:03AM +0000, Tian, Kevin wrote: >>>> From: Jason Gunthorpe<jgg@ziepe.ca> >>>> Sent: Wednesday, August 2, 2023 10:16 PM >>>> >>>> On Tue, Aug 01, 2023 at 02:31:23PM +0800, Lu Baolu wrote: >>>>> The PCI PASID enabling interface guarantees that the address space >>>>> used >>>>> by each PASID is unique. This is achieved by checking that the PCI ACS >>>>> path is enabled for the device. If the path is not enabled, then the >>>>> PASID feature cannot be used. >>>>> >>>>> if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF)) >>>>> return -EINVAL; >>>>> >>>>> The PASID array is not an attribute of the IOMMU group. It is more >>>>> natural to store the PASID array in the per-device IOMMU data. This >>>>> makes the code clearer and easier to understand. No functional changes >>>>> are intended. >>>> Is there a reason to do this? >>>> >>>> *PCI* requires the ACS/etc because PCI kind of messed up how switches >>>> handled PASID so PASID doesn't work otherwise. >>>> >>>> But there is nothing that says other bus type can't have working >>>> (non-PCI) PASID and still have device isolation issues. >>>> >>>> So unless there is a really strong reason to do this we should keep >>>> the PASID list in the group just like the domain. >>>> >>> this comes from the consensus in [1]. >>> >>> [1]https://lore.kernel.org/linux-iommu/ZAcyEzN4102gPsWC@nvidia.com/ >> That consensus was that we don't have PASID support if there is >> multi-device groups, at least in iommufd.. That makes sense. If we >> want to change the core code to enforce this that also makes sense > > In my initial plan, I had a third patch that would have enforced single- > device groups for PASID interfaces in the core. But I ultimately dropped > it because it is the fact for PCI devices, but I am not sure about other > buses although perhaps there is none. > >> But this series is just moving the array? > > So I took the first step by moving the pasid_array from iommu group to > the device.
On Fri, Aug 04, 2023 at 10:30:12AM +0800, Baolu Lu wrote: > On 2023/8/4 10:20, Baolu Lu wrote: > > On 2023/8/3 23:18, Jason Gunthorpe wrote: > > > On Thu, Aug 03, 2023 at 12:44:03AM +0000, Tian, Kevin wrote: > > > > > From: Jason Gunthorpe<jgg@ziepe.ca> > > > > > Sent: Wednesday, August 2, 2023 10:16 PM > > > > > > > > > > On Tue, Aug 01, 2023 at 02:31:23PM +0800, Lu Baolu wrote: > > > > > > The PCI PASID enabling interface guarantees that the > > > > > > address space used > > > > > > by each PASID is unique. This is achieved by checking that the PCI ACS > > > > > > path is enabled for the device. If the path is not enabled, then the > > > > > > PASID feature cannot be used. > > > > > > > > > > > > if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF)) > > > > > > return -EINVAL; > > > > > > > > > > > > The PASID array is not an attribute of the IOMMU group. It is more > > > > > > natural to store the PASID array in the per-device IOMMU data. This > > > > > > makes the code clearer and easier to understand. No functional changes > > > > > > are intended. > > > > > Is there a reason to do this? > > > > > > > > > > *PCI* requires the ACS/etc because PCI kind of messed up how switches > > > > > handled PASID so PASID doesn't work otherwise. > > > > > > > > > > But there is nothing that says other bus type can't have working > > > > > (non-PCI) PASID and still have device isolation issues. > > > > > > > > > > So unless there is a really strong reason to do this we should keep > > > > > the PASID list in the group just like the domain. > > > > > > > > > this comes from the consensus in [1]. > > > > > > > > [1]https://lore.kernel.org/linux-iommu/ZAcyEzN4102gPsWC@nvidia.com/ > > > That consensus was that we don't have PASID support if there is > > > multi-device groups, at least in iommufd.. That makes sense. If we > > > want to change the core code to enforce this that also makes sense > > > > In my initial plan, I had a third patch that would have enforced single- > > device groups for PASID interfaces in the core. But I ultimately dropped > > it because it is the fact for PCI devices, but I am not sure about other > > buses although perhaps there is none. > > > > > But this series is just moving the array? > > > > So I took the first step by moving the pasid_array from iommu group to > > the device.
The PCI PASID enabling interface guarantees that the address space used by each PASID is unique. This is achieved by checking that the PCI ACS path is enabled for the device. If the path is not enabled, then the PASID feature cannot be used. if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF)) return -EINVAL; The PASID array is not an attribute of the IOMMU group. It is more natural to store the PASID array in the per-device IOMMU data. This makes the code clearer and easier to understand. No functional changes are intended. Please help review and suggest. Lu Baolu (2): iommu: Consolidate pasid dma ownership check iommu: Move pasid array from group to device include/linux/iommu.h | 2 + drivers/iommu/iommu.c | 105 +++++++++++++++++------------------------- 2 files changed, 43 insertions(+), 64 deletions(-)