mbox series

[RFC,v1,00/15] iommu/virtio: Nested stage support with Arm

Message ID 20210115121342.15093-1-vivek.gautam@arm.com (mailing list archive)
Headers show
Series iommu/virtio: Nested stage support with Arm | expand

Message

Vivek Kumar Gautam Jan. 15, 2021, 12:13 p.m. UTC
This patch-series aims at enabling Nested stage translation in guests
using virtio-iommu as the paravirtualized iommu. The backend is supported
with Arm SMMU-v3 that provides nested stage-1 and stage-2 translation.

This series derives its purpose from various efforts happening to add
support for Shared Virtual Addressing (SVA) in host and guest. On Arm,
most of the support for SVA has already landed. The support for nested
stage translation and fault reporting to guest has been proposed [1].
The related changes required in VFIO [2] framework have also been put
forward.

This series proposes changes in virtio-iommu to program PASID tables
and related stage-1 page tables. A simple iommu-pasid-table library
is added for this purpose that interacts with vendor drivers to
allocate and populate PASID tables.
In Arm SMMUv3 we propose to pull the Context Descriptor (CD) management
code out of the arm-smmu-v3 driver and add that as a glue vendor layer
to support allocating CD tables, and populating them with right values.
These CD tables are essentially the PASID tables and contain stage-1
page table configurations too.
A request to setup these CD tables come from virtio-iommu driver using
the iommu-pasid-table library when running on Arm. The virtio-iommu
then pass these PASID tables to the host using the right virtio backend
and support in VMM.

For testing we have added necessary support in kvmtool. The changes in
kvmtool are based on virtio-iommu development branch by Jean-Philippe
Brucker [3].

The tested kernel branch contains following in the order bottom to top
on the git hash -
a) v5.11-rc3
b) arm-smmu-v3 [1] and vfio [2] changes from Eric to add nested page
   table support for Arm.
c) Smmu test engine patches from Jean-Philippe's branch [4]
d) This series
e) Domain nesting info patches [5][6][7].
f) Changes to add arm-smmu-v3 specific nesting info (to be sent to
   the list).

This kernel is tested on Neoverse reference software stack with
Fixed virtual platform. Public version of the software stack and
FVP is available here[8][9].

A big thanks to Jean-Philippe for his contributions towards this work
and for his valuable guidance.

[1] https://lore.kernel.org/linux-iommu/20201118112151.25412-1-eric.auger@redhat.com/T/
[2] https://lore.kernel.org/kvmarm/20201116110030.32335-12-eric.auger@redhat.com/T/
[3] https://jpbrucker.net/git/kvmtool/log/?h=virtio-iommu/devel
[4] https://jpbrucker.net/git/linux/log/?h=sva/smmute
[5] https://lore.kernel.org/kvm/1599734733-6431-2-git-send-email-yi.l.liu@intel.com/
[6] https://lore.kernel.org/kvm/1599734733-6431-3-git-send-email-yi.l.liu@intel.com/
[7] https://lore.kernel.org/kvm/1599734733-6431-4-git-send-email-yi.l.liu@intel.com/
[8] https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps
[9] https://git.linaro.org/landing-teams/working/arm/arm-reference-platforms.git/about/docs/rdn1edge/user-guide.rst

Jean-Philippe Brucker (6):
  iommu/virtio: Add headers for table format probing
  iommu/virtio: Add table format probing
  iommu/virtio: Add headers for binding pasid table in iommu
  iommu/virtio: Add support for INVALIDATE request
  iommu/virtio: Attach Arm PASID tables when available
  iommu/virtio: Add support for Arm LPAE page table format

Vivek Gautam (9):
  iommu/arm-smmu-v3: Create a Context Descriptor library
  iommu: Add a simple PASID table library
  iommu/arm-smmu-v3: Update drivers to work with iommu-pasid-table
  iommu/arm-smmu-v3: Update CD base address info for user-space
  iommu/arm-smmu-v3: Set sync op from consumer driver of cd-lib
  iommu: Add asid_bits to arm smmu-v3 stage1 table info
  iommu/virtio: Update table format probing header
  iommu/virtio: Prepare to add attach pasid table infrastructure
  iommu/virtio: Update fault type and reason info for viommu fault

 drivers/iommu/arm/arm-smmu-v3/Makefile        |   2 +-
 .../arm/arm-smmu-v3/arm-smmu-v3-cd-lib.c      | 283 +++++++
 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   |  16 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   | 268 +------
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h   |   4 +-
 drivers/iommu/iommu-pasid-table.h             | 140 ++++
 drivers/iommu/virtio-iommu.c                  | 692 +++++++++++++++++-
 include/uapi/linux/iommu.h                    |   2 +-
 include/uapi/linux/virtio_iommu.h             | 158 +++-
 9 files changed, 1303 insertions(+), 262 deletions(-)
 create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-cd-lib.c
 create mode 100644 drivers/iommu/iommu-pasid-table.h

Comments

Eric Auger Jan. 19, 2021, 9:03 a.m. UTC | #1
Hi Vivek,

On 1/15/21 1:13 PM, Vivek Gautam wrote:
> This patch-series aims at enabling Nested stage translation in guests
> using virtio-iommu as the paravirtualized iommu. The backend is supported
> with Arm SMMU-v3 that provides nested stage-1 and stage-2 translation.
> 
> This series derives its purpose from various efforts happening to add
> support for Shared Virtual Addressing (SVA) in host and guest. On Arm,
> most of the support for SVA has already landed. The support for nested
> stage translation and fault reporting to guest has been proposed [1].
> The related changes required in VFIO [2] framework have also been put
> forward.
> 
> This series proposes changes in virtio-iommu to program PASID tables
> and related stage-1 page tables. A simple iommu-pasid-table library
> is added for this purpose that interacts with vendor drivers to
> allocate and populate PASID tables.
> In Arm SMMUv3 we propose to pull the Context Descriptor (CD) management
> code out of the arm-smmu-v3 driver and add that as a glue vendor layer
> to support allocating CD tables, and populating them with right values.
> These CD tables are essentially the PASID tables and contain stage-1
> page table configurations too.
> A request to setup these CD tables come from virtio-iommu driver using
> the iommu-pasid-table library when running on Arm. The virtio-iommu
> then pass these PASID tables to the host using the right virtio backend
> and support in VMM.
> 
> For testing we have added necessary support in kvmtool. The changes in
> kvmtool are based on virtio-iommu development branch by Jean-Philippe
> Brucker [3].
> 
> The tested kernel branch contains following in the order bottom to top
> on the git hash -
> a) v5.11-rc3
> b) arm-smmu-v3 [1] and vfio [2] changes from Eric to add nested page
>    table support for Arm.
> c) Smmu test engine patches from Jean-Philippe's branch [4]
> d) This series
> e) Domain nesting info patches [5][6][7].
> f) Changes to add arm-smmu-v3 specific nesting info (to be sent to
>    the list).
> 
> This kernel is tested on Neoverse reference software stack with
> Fixed virtual platform. Public version of the software stack and
> FVP is available here[8][9].
> 
> A big thanks to Jean-Philippe for his contributions towards this work
> and for his valuable guidance.
> 
> [1] https://lore.kernel.org/linux-iommu/20201118112151.25412-1-eric.auger@redhat.com/T/
> [2] https://lore.kernel.org/kvmarm/20201116110030.32335-12-eric.auger@redhat.com/T/
> [3] https://jpbrucker.net/git/kvmtool/log/?h=virtio-iommu/devel
> [4] https://jpbrucker.net/git/linux/log/?h=sva/smmute
> [5] https://lore.kernel.org/kvm/1599734733-6431-2-git-send-email-yi.l.liu@intel.com/
> [6] https://lore.kernel.org/kvm/1599734733-6431-3-git-send-email-yi.l.liu@intel.com/
> [7] https://lore.kernel.org/kvm/1599734733-6431-4-git-send-email-yi.l.liu@intel.com/
> [8] https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps
> [9] https://git.linaro.org/landing-teams/working/arm/arm-reference-platforms.git/about/docs/rdn1edge/user-guide.rst

Could you share a public branch where we could find all the kernel pieces.

Thank you in advance

Best Regards

Eric
> 
> Jean-Philippe Brucker (6):
>   iommu/virtio: Add headers for table format probing
>   iommu/virtio: Add table format probing
>   iommu/virtio: Add headers for binding pasid table in iommu
>   iommu/virtio: Add support for INVALIDATE request
>   iommu/virtio: Attach Arm PASID tables when available
>   iommu/virtio: Add support for Arm LPAE page table format
> 
> Vivek Gautam (9):
>   iommu/arm-smmu-v3: Create a Context Descriptor library
>   iommu: Add a simple PASID table library
>   iommu/arm-smmu-v3: Update drivers to work with iommu-pasid-table
>   iommu/arm-smmu-v3: Update CD base address info for user-space
>   iommu/arm-smmu-v3: Set sync op from consumer driver of cd-lib
>   iommu: Add asid_bits to arm smmu-v3 stage1 table info
>   iommu/virtio: Update table format probing header
>   iommu/virtio: Prepare to add attach pasid table infrastructure
>   iommu/virtio: Update fault type and reason info for viommu fault
> 
>  drivers/iommu/arm/arm-smmu-v3/Makefile        |   2 +-
>  .../arm/arm-smmu-v3/arm-smmu-v3-cd-lib.c      | 283 +++++++
>  .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   |  16 +-
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   | 268 +------
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h   |   4 +-
>  drivers/iommu/iommu-pasid-table.h             | 140 ++++
>  drivers/iommu/virtio-iommu.c                  | 692 +++++++++++++++++-
>  include/uapi/linux/iommu.h                    |   2 +-
>  include/uapi/linux/virtio_iommu.h             | 158 +++-
>  9 files changed, 1303 insertions(+), 262 deletions(-)
>  create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-cd-lib.c
>  create mode 100644 drivers/iommu/iommu-pasid-table.h
>
Vivek Kumar Gautam Jan. 21, 2021, 5:34 p.m. UTC | #2
Hi Eric,


On 1/19/21 2:33 PM, Auger Eric wrote:
> Hi Vivek,
> 
> On 1/15/21 1:13 PM, Vivek Gautam wrote:
>> This patch-series aims at enabling Nested stage translation in guests
>> using virtio-iommu as the paravirtualized iommu. The backend is supported
>> with Arm SMMU-v3 that provides nested stage-1 and stage-2 translation.
>>
>> This series derives its purpose from various efforts happening to add
>> support for Shared Virtual Addressing (SVA) in host and guest. On Arm,
>> most of the support for SVA has already landed. The support for nested
>> stage translation and fault reporting to guest has been proposed [1].
>> The related changes required in VFIO [2] framework have also been put
>> forward.
>>
>> This series proposes changes in virtio-iommu to program PASID tables
>> and related stage-1 page tables. A simple iommu-pasid-table library
>> is added for this purpose that interacts with vendor drivers to
>> allocate and populate PASID tables.
>> In Arm SMMUv3 we propose to pull the Context Descriptor (CD) management
>> code out of the arm-smmu-v3 driver and add that as a glue vendor layer
>> to support allocating CD tables, and populating them with right values.
>> These CD tables are essentially the PASID tables and contain stage-1
>> page table configurations too.
>> A request to setup these CD tables come from virtio-iommu driver using
>> the iommu-pasid-table library when running on Arm. The virtio-iommu
>> then pass these PASID tables to the host using the right virtio backend
>> and support in VMM.
>>
>> For testing we have added necessary support in kvmtool. The changes in
>> kvmtool are based on virtio-iommu development branch by Jean-Philippe
>> Brucker [3].
>>
>> The tested kernel branch contains following in the order bottom to top
>> on the git hash -
>> a) v5.11-rc3
>> b) arm-smmu-v3 [1] and vfio [2] changes from Eric to add nested page
>>     table support for Arm.
>> c) Smmu test engine patches from Jean-Philippe's branch [4]
>> d) This series
>> e) Domain nesting info patches [5][6][7].
>> f) Changes to add arm-smmu-v3 specific nesting info (to be sent to
>>     the list).
>>
>> This kernel is tested on Neoverse reference software stack with
>> Fixed virtual platform. Public version of the software stack and
>> FVP is available here[8][9].
>>
>> A big thanks to Jean-Philippe for his contributions towards this work
>> and for his valuable guidance.
>>
>> [1] https://lore.kernel.org/linux-iommu/20201118112151.25412-1-eric.auger@redhat.com/T/
>> [2] https://lore.kernel.org/kvmarm/20201116110030.32335-12-eric.auger@redhat.com/T/
>> [3] https://jpbrucker.net/git/kvmtool/log/?h=virtio-iommu/devel
>> [4] https://jpbrucker.net/git/linux/log/?h=sva/smmute
>> [5] https://lore.kernel.org/kvm/1599734733-6431-2-git-send-email-yi.l.liu@intel.com/
>> [6] https://lore.kernel.org/kvm/1599734733-6431-3-git-send-email-yi.l.liu@intel.com/
>> [7] https://lore.kernel.org/kvm/1599734733-6431-4-git-send-email-yi.l.liu@intel.com/
>> [8] https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps
>> [9] https://git.linaro.org/landing-teams/working/arm/arm-reference-platforms.git/about/docs/rdn1edge/user-guide.rst
> 
> Could you share a public branch where we could find all the kernel pieces.
> 
> Thank you in advance

Apologies for the delay. It took a bit of time to sort things out for a 
public branch.
The branch is available in my github now. Please have a look.

https://github.com/vivek-arm/linux/tree/5.11-rc3-nested-pgtbl-arm-smmuv3-virtio-iommu


Thanks and regards
Vivek

> 
> Best Regards
> 
> Eric
>>
>> Jean-Philippe Brucker (6):
>>    iommu/virtio: Add headers for table format probing
>>    iommu/virtio: Add table format probing
>>    iommu/virtio: Add headers for binding pasid table in iommu
>>    iommu/virtio: Add support for INVALIDATE request
>>    iommu/virtio: Attach Arm PASID tables when available
>>    iommu/virtio: Add support for Arm LPAE page table format
>>
>> Vivek Gautam (9):
>>    iommu/arm-smmu-v3: Create a Context Descriptor library
>>    iommu: Add a simple PASID table library
>>    iommu/arm-smmu-v3: Update drivers to work with iommu-pasid-table
>>    iommu/arm-smmu-v3: Update CD base address info for user-space
>>    iommu/arm-smmu-v3: Set sync op from consumer driver of cd-lib
>>    iommu: Add asid_bits to arm smmu-v3 stage1 table info
>>    iommu/virtio: Update table format probing header
>>    iommu/virtio: Prepare to add attach pasid table infrastructure
>>    iommu/virtio: Update fault type and reason info for viommu fault
>>
>>   drivers/iommu/arm/arm-smmu-v3/Makefile        |   2 +-
>>   .../arm/arm-smmu-v3/arm-smmu-v3-cd-lib.c      | 283 +++++++
>>   .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   |  16 +-
>>   drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   | 268 +------
>>   drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h   |   4 +-
>>   drivers/iommu/iommu-pasid-table.h             | 140 ++++
>>   drivers/iommu/virtio-iommu.c                  | 692 +++++++++++++++++-
>>   include/uapi/linux/iommu.h                    |   2 +-
>>   include/uapi/linux/virtio_iommu.h             | 158 +++-
>>   9 files changed, 1303 insertions(+), 262 deletions(-)
>>   create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-cd-lib.c
>>   create mode 100644 drivers/iommu/iommu-pasid-table.h
>>
>
Shameerali Kolothum Thodi Jan. 22, 2021, 3:49 p.m. UTC | #3
Hi Vivek,

> -----Original Message-----
> From: Vivek Kumar Gautam [mailto:vivek.gautam@arm.com]
> Sent: 21 January 2021 17:34
> To: Auger Eric <eric.auger@redhat.com>; linux-kernel@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; iommu@lists.linux-foundation.org;
> virtualization@lists.linux-foundation.org
> Cc: joro@8bytes.org; will.deacon@arm.com; mst@redhat.com;
> robin.murphy@arm.com; jean-philippe@linaro.org;
> alex.williamson@redhat.com; kevin.tian@intel.com;
> jacob.jun.pan@linux.intel.com; yi.l.liu@intel.com; lorenzo.pieralisi@arm.com;
> Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Subject: Re: [PATCH RFC v1 00/15] iommu/virtio: Nested stage support with
> Arm
> 
> Hi Eric,
> 
> 
> On 1/19/21 2:33 PM, Auger Eric wrote:
> > Hi Vivek,
> >
> > On 1/15/21 1:13 PM, Vivek Gautam wrote:
> >> This patch-series aims at enabling Nested stage translation in guests
> >> using virtio-iommu as the paravirtualized iommu. The backend is
> >> supported with Arm SMMU-v3 that provides nested stage-1 and stage-2
> translation.
> >>
> >> This series derives its purpose from various efforts happening to add
> >> support for Shared Virtual Addressing (SVA) in host and guest. On
> >> Arm, most of the support for SVA has already landed. The support for
> >> nested stage translation and fault reporting to guest has been proposed [1].
> >> The related changes required in VFIO [2] framework have also been put
> >> forward.
> >>
> >> This series proposes changes in virtio-iommu to program PASID tables
> >> and related stage-1 page tables. A simple iommu-pasid-table library
> >> is added for this purpose that interacts with vendor drivers to
> >> allocate and populate PASID tables.
> >> In Arm SMMUv3 we propose to pull the Context Descriptor (CD)
> >> management code out of the arm-smmu-v3 driver and add that as a glue
> >> vendor layer to support allocating CD tables, and populating them with right
> values.
> >> These CD tables are essentially the PASID tables and contain stage-1
> >> page table configurations too.
> >> A request to setup these CD tables come from virtio-iommu driver
> >> using the iommu-pasid-table library when running on Arm. The
> >> virtio-iommu then pass these PASID tables to the host using the right
> >> virtio backend and support in VMM.
> >>
> >> For testing we have added necessary support in kvmtool. The changes
> >> in kvmtool are based on virtio-iommu development branch by
> >> Jean-Philippe Brucker [3].
> >>
> >> The tested kernel branch contains following in the order bottom to
> >> top on the git hash -
> >> a) v5.11-rc3
> >> b) arm-smmu-v3 [1] and vfio [2] changes from Eric to add nested page
> >>     table support for Arm.
> >> c) Smmu test engine patches from Jean-Philippe's branch [4]
> >> d) This series
> >> e) Domain nesting info patches [5][6][7].
> >> f) Changes to add arm-smmu-v3 specific nesting info (to be sent to
> >>     the list).
> >>
> >> This kernel is tested on Neoverse reference software stack with Fixed
> >> virtual platform. Public version of the software stack and FVP is
> >> available here[8][9].
> >>
> >> A big thanks to Jean-Philippe for his contributions towards this work
> >> and for his valuable guidance.
> >>
> >> [1]
> >> https://lore.kernel.org/linux-iommu/20201118112151.25412-1-eric.auger
> >> @redhat.com/T/ [2]
> >>
> https://lore.kernel.org/kvmarm/20201116110030.32335-12-eric.auger@red
> >> hat.com/T/ [3]
> >> https://jpbrucker.net/git/kvmtool/log/?h=virtio-iommu/devel
> >> [4] https://jpbrucker.net/git/linux/log/?h=sva/smmute
> >> [5]
> >> https://lore.kernel.org/kvm/1599734733-6431-2-git-send-email-yi.l.liu
> >> @intel.com/ [6]
> >> https://lore.kernel.org/kvm/1599734733-6431-3-git-send-email-yi.l.liu
> >> @intel.com/ [7]
> >> https://lore.kernel.org/kvm/1599734733-6431-4-git-send-email-yi.l.liu
> >> @intel.com/ [8]
> >> https://developer.arm.com/tools-and-software/open-source-software/arm
> >> -platforms-software/arm-ecosystem-fvps
> >> [9]
> >> https://git.linaro.org/landing-teams/working/arm/arm-reference-platfo
> >> rms.git/about/docs/rdn1edge/user-guide.rst
> >
> > Could you share a public branch where we could find all the kernel pieces.
> >
> > Thank you in advance
> 
> Apologies for the delay. It took a bit of time to sort things out for a public
> branch.
> The branch is available in my github now. Please have a look.
> 
> https://github.com/vivek-arm/linux/tree/5.11-rc3-nested-pgtbl-arm-smmuv3-vi
> rtio-iommu

Thanks for this. Do you have a corresponding kvmtool branch mentioned above as public?

Thanks,
Shameer
Eric Auger Jan. 25, 2021, 8:43 a.m. UTC | #4
Hi Vivek,

On 1/21/21 6:34 PM, Vivek Kumar Gautam wrote:
> Hi Eric,
> 
> 
> On 1/19/21 2:33 PM, Auger Eric wrote:
>> Hi Vivek,
>>
>> On 1/15/21 1:13 PM, Vivek Gautam wrote:
>>> This patch-series aims at enabling Nested stage translation in guests
>>> using virtio-iommu as the paravirtualized iommu. The backend is
>>> supported
>>> with Arm SMMU-v3 that provides nested stage-1 and stage-2 translation.
>>>
>>> This series derives its purpose from various efforts happening to add
>>> support for Shared Virtual Addressing (SVA) in host and guest. On Arm,
>>> most of the support for SVA has already landed. The support for nested
>>> stage translation and fault reporting to guest has been proposed [1].
>>> The related changes required in VFIO [2] framework have also been put
>>> forward.
>>>
>>> This series proposes changes in virtio-iommu to program PASID tables
>>> and related stage-1 page tables. A simple iommu-pasid-table library
>>> is added for this purpose that interacts with vendor drivers to
>>> allocate and populate PASID tables.
>>> In Arm SMMUv3 we propose to pull the Context Descriptor (CD) management
>>> code out of the arm-smmu-v3 driver and add that as a glue vendor layer
>>> to support allocating CD tables, and populating them with right values.
>>> These CD tables are essentially the PASID tables and contain stage-1
>>> page table configurations too.
>>> A request to setup these CD tables come from virtio-iommu driver using
>>> the iommu-pasid-table library when running on Arm. The virtio-iommu
>>> then pass these PASID tables to the host using the right virtio backend
>>> and support in VMM.
>>>
>>> For testing we have added necessary support in kvmtool. The changes in
>>> kvmtool are based on virtio-iommu development branch by Jean-Philippe
>>> Brucker [3].
>>>
>>> The tested kernel branch contains following in the order bottom to top
>>> on the git hash -
>>> a) v5.11-rc3
>>> b) arm-smmu-v3 [1] and vfio [2] changes from Eric to add nested page
>>>     table support for Arm.
>>> c) Smmu test engine patches from Jean-Philippe's branch [4]
>>> d) This series
>>> e) Domain nesting info patches [5][6][7].
>>> f) Changes to add arm-smmu-v3 specific nesting info (to be sent to
>>>     the list).
>>>
>>> This kernel is tested on Neoverse reference software stack with
>>> Fixed virtual platform. Public version of the software stack and
>>> FVP is available here[8][9].
>>>
>>> A big thanks to Jean-Philippe for his contributions towards this work
>>> and for his valuable guidance.
>>>
>>> [1]
>>> https://lore.kernel.org/linux-iommu/20201118112151.25412-1-eric.auger@redhat.com/T/
>>>
>>> [2]
>>> https://lore.kernel.org/kvmarm/20201116110030.32335-12-eric.auger@redhat.com/T/
>>>
>>> [3] https://jpbrucker.net/git/kvmtool/log/?h=virtio-iommu/devel
>>> [4] https://jpbrucker.net/git/linux/log/?h=sva/smmute
>>> [5]
>>> https://lore.kernel.org/kvm/1599734733-6431-2-git-send-email-yi.l.liu@intel.com/
>>>
>>> [6]
>>> https://lore.kernel.org/kvm/1599734733-6431-3-git-send-email-yi.l.liu@intel.com/
>>>
>>> [7]
>>> https://lore.kernel.org/kvm/1599734733-6431-4-git-send-email-yi.l.liu@intel.com/
>>>
>>> [8]
>>> https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps
>>>
>>> [9]
>>> https://git.linaro.org/landing-teams/working/arm/arm-reference-platforms.git/about/docs/rdn1edge/user-guide.rst
>>>
>>
>> Could you share a public branch where we could find all the kernel
>> pieces.
>>
>> Thank you in advance
> 
> Apologies for the delay. It took a bit of time to sort things out for a
> public branch.
> The branch is available in my github now. Please have a look.
> 
> https://github.com/vivek-arm/linux/tree/5.11-rc3-nested-pgtbl-arm-smmuv3-virtio-iommu

no problem. Thank you for the link.

Best Regards

Eric
> 
> 
> 
> Thanks and regards
> Vivek
> 
>>
>> Best Regards
>>
>> Eric
>>>
>>> Jean-Philippe Brucker (6):
>>>    iommu/virtio: Add headers for table format probing
>>>    iommu/virtio: Add table format probing
>>>    iommu/virtio: Add headers for binding pasid table in iommu
>>>    iommu/virtio: Add support for INVALIDATE request
>>>    iommu/virtio: Attach Arm PASID tables when available
>>>    iommu/virtio: Add support for Arm LPAE page table format
>>>
>>> Vivek Gautam (9):
>>>    iommu/arm-smmu-v3: Create a Context Descriptor library
>>>    iommu: Add a simple PASID table library
>>>    iommu/arm-smmu-v3: Update drivers to work with iommu-pasid-table
>>>    iommu/arm-smmu-v3: Update CD base address info for user-space
>>>    iommu/arm-smmu-v3: Set sync op from consumer driver of cd-lib
>>>    iommu: Add asid_bits to arm smmu-v3 stage1 table info
>>>    iommu/virtio: Update table format probing header
>>>    iommu/virtio: Prepare to add attach pasid table infrastructure
>>>    iommu/virtio: Update fault type and reason info for viommu fault
>>>
>>>   drivers/iommu/arm/arm-smmu-v3/Makefile        |   2 +-
>>>   .../arm/arm-smmu-v3/arm-smmu-v3-cd-lib.c      | 283 +++++++
>>>   .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   |  16 +-
>>>   drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   | 268 +------
>>>   drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h   |   4 +-
>>>   drivers/iommu/iommu-pasid-table.h             | 140 ++++
>>>   drivers/iommu/virtio-iommu.c                  | 692 +++++++++++++++++-
>>>   include/uapi/linux/iommu.h                    |   2 +-
>>>   include/uapi/linux/virtio_iommu.h             | 158 +++-
>>>   9 files changed, 1303 insertions(+), 262 deletions(-)
>>>   create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-cd-lib.c
>>>   create mode 100644 drivers/iommu/iommu-pasid-table.h
>>>
>>
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
Vivek Kumar Gautam Jan. 25, 2021, 12:55 p.m. UTC | #5
Hi Shameer,


On 1/22/21 9:19 PM, Shameerali Kolothum Thodi wrote:
> Hi Vivek,
> 
>> -----Original Message-----
>> From: Vivek Kumar Gautam [mailto:vivek.gautam@arm.com]
>> Sent: 21 January 2021 17:34
>> To: Auger Eric <eric.auger@redhat.com>; linux-kernel@vger.kernel.org;
>> linux-arm-kernel@lists.infradead.org; iommu@lists.linux-foundation.org;
>> virtualization@lists.linux-foundation.org
>> Cc: joro@8bytes.org; will.deacon@arm.com; mst@redhat.com;
>> robin.murphy@arm.com; jean-philippe@linaro.org;
>> alex.williamson@redhat.com; kevin.tian@intel.com;
>> jacob.jun.pan@linux.intel.com; yi.l.liu@intel.com; lorenzo.pieralisi@arm.com;
>> Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
>> Subject: Re: [PATCH RFC v1 00/15] iommu/virtio: Nested stage support with
>> Arm
>>
>> Hi Eric,
>>
>>
>> On 1/19/21 2:33 PM, Auger Eric wrote:
>>> Hi Vivek,
>>>
>>> On 1/15/21 1:13 PM, Vivek Gautam wrote:
>>>> This patch-series aims at enabling Nested stage translation in guests
>>>> using virtio-iommu as the paravirtualized iommu. The backend is
>>>> supported with Arm SMMU-v3 that provides nested stage-1 and stage-2
>> translation.
>>>>
>>>> This series derives its purpose from various efforts happening to add
>>>> support for Shared Virtual Addressing (SVA) in host and guest. On
>>>> Arm, most of the support for SVA has already landed. The support for
>>>> nested stage translation and fault reporting to guest has been proposed [1].
>>>> The related changes required in VFIO [2] framework have also been put
>>>> forward.
>>>>
>>>> This series proposes changes in virtio-iommu to program PASID tables
>>>> and related stage-1 page tables. A simple iommu-pasid-table library
>>>> is added for this purpose that interacts with vendor drivers to
>>>> allocate and populate PASID tables.
>>>> In Arm SMMUv3 we propose to pull the Context Descriptor (CD)
>>>> management code out of the arm-smmu-v3 driver and add that as a glue
>>>> vendor layer to support allocating CD tables, and populating them with right
>> values.
>>>> These CD tables are essentially the PASID tables and contain stage-1
>>>> page table configurations too.
>>>> A request to setup these CD tables come from virtio-iommu driver
>>>> using the iommu-pasid-table library when running on Arm. The
>>>> virtio-iommu then pass these PASID tables to the host using the right
>>>> virtio backend and support in VMM.
>>>>
>>>> For testing we have added necessary support in kvmtool. The changes
>>>> in kvmtool are based on virtio-iommu development branch by
>>>> Jean-Philippe Brucker [3].
>>>>
>>>> The tested kernel branch contains following in the order bottom to
>>>> top on the git hash -
>>>> a) v5.11-rc3
>>>> b) arm-smmu-v3 [1] and vfio [2] changes from Eric to add nested page
>>>>      table support for Arm.
>>>> c) Smmu test engine patches from Jean-Philippe's branch [4]
>>>> d) This series
>>>> e) Domain nesting info patches [5][6][7].
>>>> f) Changes to add arm-smmu-v3 specific nesting info (to be sent to
>>>>      the list).
>>>>
>>>> This kernel is tested on Neoverse reference software stack with Fixed
>>>> virtual platform. Public version of the software stack and FVP is
>>>> available here[8][9].
>>>>
>>>> A big thanks to Jean-Philippe for his contributions towards this work
>>>> and for his valuable guidance.
>>>>
>>>> [1]
>>>> https://lore.kernel.org/linux-iommu/20201118112151.25412-1-eric.auger
>>>> @redhat.com/T/ [2]
>>>>
>> https://lore.kernel.org/kvmarm/20201116110030.32335-12-eric.auger@red
>>>> hat.com/T/ [3]
>>>> https://jpbrucker.net/git/kvmtool/log/?h=virtio-iommu/devel
>>>> [4] https://jpbrucker.net/git/linux/log/?h=sva/smmute
>>>> [5]
>>>> https://lore.kernel.org/kvm/1599734733-6431-2-git-send-email-yi.l.liu
>>>> @intel.com/ [6]
>>>> https://lore.kernel.org/kvm/1599734733-6431-3-git-send-email-yi.l.liu
>>>> @intel.com/ [7]
>>>> https://lore.kernel.org/kvm/1599734733-6431-4-git-send-email-yi.l.liu
>>>> @intel.com/ [8]
>>>> https://developer.arm.com/tools-and-software/open-source-software/arm
>>>> -platforms-software/arm-ecosystem-fvps
>>>> [9]
>>>> https://git.linaro.org/landing-teams/working/arm/arm-reference-platfo
>>>> rms.git/about/docs/rdn1edge/user-guide.rst
>>>
>>> Could you share a public branch where we could find all the kernel pieces.
>>>
>>> Thank you in advance
>>
>> Apologies for the delay. It took a bit of time to sort things out for a public
>> branch.
>> The branch is available in my github now. Please have a look.
>>
>> https://github.com/vivek-arm/linux/tree/5.11-rc3-nested-pgtbl-arm-smmuv3-vi
>> rtio-iommu
>  > Thanks for this. Do you have a corresponding kvmtool branch mentioned 
above as public?

Thanks for showing interest. I will publish the kvmtool branch asap. 
Though the current development is based on Jean's branch for 
virtio-iommu [1], I plan to rebase the changes to master soon.

Thanks & regards
Vivek

[1] https://jpbrucker.net/git/kvmtool/log/?h=virtio-iommu/devel
> 
> Thanks,
> Shameer
>