| Message ID | 20210629143621.907831-1-eric.auger@redhat.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | avocado-qemu: New SMMUv3 and intel IOMMU tests | expand |
Hi Cleber, all, On 6/29/21 4:36 PM, Eric Auger wrote: > This series adds ARM SMMU and Intel IOMMU functional > tests using Fedora cloud-init images. > > ARM SMMU tests feature guests with and without RIL > (range invalidation support) using respectively fedora 33 > and 31. For each, we test the protection of virtio-net-pci > and virtio-block-pci devices. Also strict=no and passthrough > modes are tested. So there is a total of 6 tests. > > The series applies on top of Cleber's series: > - [PATCH 0/3] Acceptance Tests: support choosing specific > > Note: > - SMMU tests 2, 3, 5, 6 (resp. test_smmu_noril_passthrough and > test_smmu_noril_nostrict) pass but the log reports: > "WARN: Test passed but there were warnings during execution." > This seems due to the lack of hash when fetching the kernel and > initrd through fetch_asset(): > WARNI| No hash provided. Cannot check the asset file integrity. I wanted to emphasize that point and wondered how we could fix that issue. Looks a pity the tests get tagged as WARN due to a lack of sha1. Any advice? Best Regards Eric > > History: > v3 -> v4: > - I added Wainer's refactoring of KNOWN_DISTROS > into a class (last patch) and took into account his comments. > > v2 -> v3: > - Added Intel IOMMU tests were added. Different > operating modes are tested such as strict, caching mode, pt. > > Best Regards > > Eric > > The series and its dependencies can be found at: > https://github.com/eauger/qemu/tree/avocado-qemu-v4 > > Eric Auger (3): > Acceptance Tests: Add default kernel params and pxeboot url to the > KNOWN_DISTROS collection > avocado_qemu: Add SMMUv3 tests > avocado_qemu: Add Intel iommu tests > > Wainer dos Santos Moschetta (1): > avocado_qemu: Fix KNOWN_DISTROS map into the LinuxDistro class > > tests/acceptance/avocado_qemu/__init__.py | 118 +++++++++++++------ > tests/acceptance/intel_iommu.py | 115 +++++++++++++++++++ > tests/acceptance/smmu.py | 132 ++++++++++++++++++++++ > 3 files changed, 332 insertions(+), 33 deletions(-) > create mode 100644 tests/acceptance/intel_iommu.py > create mode 100644 tests/acceptance/smmu.py >
On Tue, Jun 29, 2021 at 5:17 PM Eric Auger <eric.auger@redhat.com> wrote: > > Hi Cleber, all, > > On 6/29/21 4:36 PM, Eric Auger wrote: > > This series adds ARM SMMU and Intel IOMMU functional > > tests using Fedora cloud-init images. > > > > ARM SMMU tests feature guests with and without RIL > > (range invalidation support) using respectively fedora 33 > > and 31. For each, we test the protection of virtio-net-pci > > and virtio-block-pci devices. Also strict=no and passthrough > > modes are tested. So there is a total of 6 tests. > > > > The series applies on top of Cleber's series: > > - [PATCH 0/3] Acceptance Tests: support choosing specific > > > > Note: > > - SMMU tests 2, 3, 5, 6 (resp. test_smmu_noril_passthrough and > > test_smmu_noril_nostrict) pass but the log reports: > > "WARN: Test passed but there were warnings during execution." > > This seems due to the lack of hash when fetching the kernel and > > initrd through fetch_asset(): > > WARNI| No hash provided. Cannot check the asset file integrity. > I wanted to emphasize that point and wondered how we could fix that > issue. Looks a pity the tests get tagged as WARN due to a lack of sha1. > Any advice? Hi Eric, We had that discussion some weeks ago regarding the WARN status of a test when the file hash is not provided for the fetch call. We agreed that a WARN is not a harmful status, and it would be okay. When we got the request to add the message regarding a missing hash of a downloaded file, we concluded that it would not make sense to set it like a normal message in the logs because no one would open the logs and see the message if the test succeed. If you think a WARN may be considered a harmful status, let us know, and we can try to adjust it so that users see the message when a hash is not provided without setting the test status as WARN. On the other hand, you can always add the hash if you have access to it. I hope it helps somehow, Willian > > Best Regards > > Eric > > > > History: > > v3 -> v4: > > - I added Wainer's refactoring of KNOWN_DISTROS > > into a class (last patch) and took into account his comments. > > > > v2 -> v3: > > - Added Intel IOMMU tests were added. Different > > operating modes are tested such as strict, caching mode, pt. > > > > Best Regards > > > > Eric > > > > The series and its dependencies can be found at: > > https://github.com/eauger/qemu/tree/avocado-qemu-v4 > > > > Eric Auger (3): > > Acceptance Tests: Add default kernel params and pxeboot url to the > > KNOWN_DISTROS collection > > avocado_qemu: Add SMMUv3 tests > > avocado_qemu: Add Intel iommu tests > > > > Wainer dos Santos Moschetta (1): > > avocado_qemu: Fix KNOWN_DISTROS map into the LinuxDistro class > > > > tests/acceptance/avocado_qemu/__init__.py | 118 +++++++++++++------ > > tests/acceptance/intel_iommu.py | 115 +++++++++++++++++++ > > tests/acceptance/smmu.py | 132 ++++++++++++++++++++++ > > 3 files changed, 332 insertions(+), 33 deletions(-) > > create mode 100644 tests/acceptance/intel_iommu.py > > create mode 100644 tests/acceptance/smmu.py > > >
Hi William, On 6/29/21 10:38 PM, Willian Rampazzo wrote: > On Tue, Jun 29, 2021 at 5:17 PM Eric Auger <eric.auger@redhat.com> wrote: >> Hi Cleber, all, >> >> On 6/29/21 4:36 PM, Eric Auger wrote: >>> This series adds ARM SMMU and Intel IOMMU functional >>> tests using Fedora cloud-init images. >>> >>> ARM SMMU tests feature guests with and without RIL >>> (range invalidation support) using respectively fedora 33 >>> and 31. For each, we test the protection of virtio-net-pci >>> and virtio-block-pci devices. Also strict=no and passthrough >>> modes are tested. So there is a total of 6 tests. >>> >>> The series applies on top of Cleber's series: >>> - [PATCH 0/3] Acceptance Tests: support choosing specific >>> >>> Note: >>> - SMMU tests 2, 3, 5, 6 (resp. test_smmu_noril_passthrough and >>> test_smmu_noril_nostrict) pass but the log reports: >>> "WARN: Test passed but there were warnings during execution." >>> This seems due to the lack of hash when fetching the kernel and >>> initrd through fetch_asset(): >>> WARNI| No hash provided. Cannot check the asset file integrity. >> I wanted to emphasize that point and wondered how we could fix that >> issue. Looks a pity the tests get tagged as WARN due to a lack of sha1. >> Any advice? > Hi Eric, > > We had that discussion some weeks ago regarding the WARN status of a > test when the file hash is not provided for the fetch call. We agreed > that a WARN is not a harmful status, and it would be okay. > > When we got the request to add the message regarding a missing hash of > a downloaded file, we concluded that it would not make sense to set it > like a normal message in the logs because no one would open the logs > and see the message if the test succeed. > > If you think a WARN may be considered a harmful status, let us know, > and we can try to adjust it so that users see the message when a hash > is not provided without setting the test status as WARN. > > On the other hand, you can always add the hash if you have access to it. > > I hope it helps somehow, It does, thank you for your reply. Maybe we should tag the tests with warnings as PASS in the last summary RESULT line and keep the individual test result lines as they are. Currently it is not obvious the tests has passed if you look at the last line. See my logs below. (1/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril: PASS (106.83 s) (2/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril_passthrough: WARN: Test passed but there were warnings during execution. Check the log for details. (112.23 s) (3/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril_nostrict: WARN: Test passed but there were warnings during execution. Check the log for details. (110.77 s) (4/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril: PASS (135.18 s) (5/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril_passthrough: WARN: Test passed but there were warnings during execution. Check the log for details. (115.98 s) (6/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril_nostrict: WARN: Test passed but there were warnings during execution. Check the log for details. (137.16 s) RESULTS : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 4 | INTERRUPT 0 | CANCEL 0 Thanks Eric > > Willian > >> Best Regards >> >> Eric >>> History: >>> v3 -> v4: >>> - I added Wainer's refactoring of KNOWN_DISTROS >>> into a class (last patch) and took into account his comments. >>> >>> v2 -> v3: >>> - Added Intel IOMMU tests were added. Different >>> operating modes are tested such as strict, caching mode, pt. >>> >>> Best Regards >>> >>> Eric >>> >>> The series and its dependencies can be found at: >>> https://github.com/eauger/qemu/tree/avocado-qemu-v4 >>> >>> Eric Auger (3): >>> Acceptance Tests: Add default kernel params and pxeboot url to the >>> KNOWN_DISTROS collection >>> avocado_qemu: Add SMMUv3 tests >>> avocado_qemu: Add Intel iommu tests >>> >>> Wainer dos Santos Moschetta (1): >>> avocado_qemu: Fix KNOWN_DISTROS map into the LinuxDistro class >>> >>> tests/acceptance/avocado_qemu/__init__.py | 118 +++++++++++++------ >>> tests/acceptance/intel_iommu.py | 115 +++++++++++++++++++ >>> tests/acceptance/smmu.py | 132 ++++++++++++++++++++++ >>> 3 files changed, 332 insertions(+), 33 deletions(-) >>> create mode 100644 tests/acceptance/intel_iommu.py >>> create mode 100644 tests/acceptance/smmu.py >>>
Hi, On 6/29/21 5:17 PM, Eric Auger wrote: > Hi Cleber, all, > > On 6/29/21 4:36 PM, Eric Auger wrote: >> This series adds ARM SMMU and Intel IOMMU functional >> tests using Fedora cloud-init images. >> >> ARM SMMU tests feature guests with and without RIL >> (range invalidation support) using respectively fedora 33 >> and 31. For each, we test the protection of virtio-net-pci >> and virtio-block-pci devices. Also strict=no and passthrough >> modes are tested. So there is a total of 6 tests. >> >> The series applies on top of Cleber's series: >> - [PATCH 0/3] Acceptance Tests: support choosing specific >> >> Note: >> - SMMU tests 2, 3, 5, 6 (resp. test_smmu_noril_passthrough and >> test_smmu_noril_nostrict) pass but the log reports: >> "WARN: Test passed but there were warnings during execution." >> This seems due to the lack of hash when fetching the kernel and >> initrd through fetch_asset(): >> WARNI| No hash provided. Cannot check the asset file integrity. > I wanted to emphasize that point and wondered how we could fix that > issue. Looks a pity the tests get tagged as WARN due to a lack of sha1. > Any advice? As Willian mentioned somewhere, to supress the WARN you can pass the kernel and initrd checksums (sha1) to the fetch_asset() method. Below is an draft implementation. It would need to fill out the remaining checksums and adjust the `smmu.py` tests. - Wainer ---- diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py index 00eb0bfcc8..83637e2654 100644 --- a/tests/acceptance/avocado_qemu/__init__.py +++ b/tests/acceptance/avocado_qemu/__init__.py @@ -312,6 +312,8 @@ class LinuxDistro: {'checksum': 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0', 'pxeboot_url': "https://archives.fedoraproject.org/pub/archive/fedora/" "linux/releases/31/Everything/x86_64/os/images/pxeboot/", + 'pxeboot_initrd_chksum': 'dd0340a1b39bd28f88532babd4581c67649ec5b1', + 'pxeboot_vmlinuz_chksum': '5b6f6876e1b5bda314f93893271da0d5777b1f3c', 'kernel_params': "root=UUID=b1438b9b-2cab-4065-a99a-08a96687f73c ro " "no_timer_check net.ifnames=0 " "console=tty1 console=ttyS0,115200n8"}, @@ -371,6 +373,16 @@ def pxeboot_url(self): """Gets the repository url where pxeboot files can be found""" return self._info.get('pxeboot_url', None) + @property + def pxeboot_initrd_chksum(self): + """Gets the pxeboot initrd file checksum""" + return self._info.get('pxeboot_initrd_chksum', None) + + @property + def pxeboot_vmlinuz_chksum(self): + """Gets the pxeboot vmlinuz file checksum""" + return self._info.get('pxeboot_vmlinuz_chksum', None) + @property def checksum(self): """Gets the cloud-image file checksum""" diff --git a/tests/acceptance/intel_iommu.py b/tests/acceptance/intel_iommu.py index bf8dea6e4f..a2f38ee2e9 100644 --- a/tests/acceptance/intel_iommu.py +++ b/tests/acceptance/intel_iommu.py @@ -55,8 +55,10 @@ def common_vm_setup(self, custom_kernel=None): kernel_url = self.distro.pxeboot_url + 'vmlinuz' initrd_url = self.distro.pxeboot_url + 'initrd.img' - self.kernel_path = self.fetch_asset(kernel_url) - self.initrd_path = self.fetch_asset(initrd_url) + self.kernel_path = self.fetch_asset(kernel_url, + asset_hash=self.distro.pxeboot_vmlinuz_chksum) + self.initrd_path = self.fetch_asset(initrd_url, + asset_hash=self.distro.pxeboot_initrd_chksum) def run_and_check(self): if self.kernel_path: > > Best Regards > > Eric >> History: >> v3 -> v4: >> - I added Wainer's refactoring of KNOWN_DISTROS >> into a class (last patch) and took into account his comments. >> >> v2 -> v3: >> - Added Intel IOMMU tests were added. Different >> operating modes are tested such as strict, caching mode, pt. >> >> Best Regards >> >> Eric >> >> The series and its dependencies can be found at: >> https://github.com/eauger/qemu/tree/avocado-qemu-v4 >> >> Eric Auger (3): >> Acceptance Tests: Add default kernel params and pxeboot url to the >> KNOWN_DISTROS collection >> avocado_qemu: Add SMMUv3 tests >> avocado_qemu: Add Intel iommu tests >> >> Wainer dos Santos Moschetta (1): >> avocado_qemu: Fix KNOWN_DISTROS map into the LinuxDistro class >> >> tests/acceptance/avocado_qemu/__init__.py | 118 +++++++++++++------ >> tests/acceptance/intel_iommu.py | 115 +++++++++++++++++++ >> tests/acceptance/smmu.py | 132 ++++++++++++++++++++++ >> 3 files changed, 332 insertions(+), 33 deletions(-) >> create mode 100644 tests/acceptance/intel_iommu.py >> create mode 100644 tests/acceptance/smmu.py >>
Hi Wainer, On 7/1/21 1:22 AM, Wainer dos Santos Moschetta wrote: > Hi, > > On 6/29/21 5:17 PM, Eric Auger wrote: >> Hi Cleber, all, >> >> On 6/29/21 4:36 PM, Eric Auger wrote: >>> This series adds ARM SMMU and Intel IOMMU functional >>> tests using Fedora cloud-init images. >>> >>> ARM SMMU tests feature guests with and without RIL >>> (range invalidation support) using respectively fedora 33 >>> and 31. For each, we test the protection of virtio-net-pci >>> and virtio-block-pci devices. Also strict=no and passthrough >>> modes are tested. So there is a total of 6 tests. >>> >>> The series applies on top of Cleber's series: >>> - [PATCH 0/3] Acceptance Tests: support choosing specific >>> >>> Note: >>> - SMMU tests 2, 3, 5, 6 (resp. test_smmu_noril_passthrough and >>> test_smmu_noril_nostrict) pass but the log reports: >>> "WARN: Test passed but there were warnings during execution." >>> This seems due to the lack of hash when fetching the kernel and >>> initrd through fetch_asset(): >>> WARNI| No hash provided. Cannot check the asset file integrity. >> I wanted to emphasize that point and wondered how we could fix that >> issue. Looks a pity the tests get tagged as WARN due to a lack of sha1. >> Any advice? > > As Willian mentioned somewhere, to supress the WARN you can pass the > kernel and initrd checksums (sha1) to the fetch_asset() method. > > Below is an draft implementation. It would need to fill out the > remaining checksums and adjust the `smmu.py` tests. > > - Wainer > > ---- > > diff --git a/tests/acceptance/avocado_qemu/__init__.py > b/tests/acceptance/avocado_qemu/__init__.py > index 00eb0bfcc8..83637e2654 100644 > --- a/tests/acceptance/avocado_qemu/__init__.py > +++ b/tests/acceptance/avocado_qemu/__init__.py > @@ -312,6 +312,8 @@ class LinuxDistro: > {'checksum': > 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0', > 'pxeboot_url': > "https://archives.fedoraproject.org/pub/archive/fedora/" > "linux/releases/31/Everything/x86_64/os/images/pxeboot/", > + 'pxeboot_initrd_chksum': > 'dd0340a1b39bd28f88532babd4581c67649ec5b1', > + 'pxeboot_vmlinuz_chksum': > '5b6f6876e1b5bda314f93893271da0d5777b1f3c', where did you get the checksum? I don't see any at the URL? Did you generate it yourself? Thanks Eric > 'kernel_params': > "root=UUID=b1438b9b-2cab-4065-a99a-08a96687f73c ro " > "no_timer_check net.ifnames=0 " > "console=tty1 console=ttyS0,115200n8"}, > @@ -371,6 +373,16 @@ def pxeboot_url(self): > """Gets the repository url where pxeboot files can be found""" > return self._info.get('pxeboot_url', None) > > + @property > + def pxeboot_initrd_chksum(self): > + """Gets the pxeboot initrd file checksum""" > + return self._info.get('pxeboot_initrd_chksum', None) > + > + @property > + def pxeboot_vmlinuz_chksum(self): > + """Gets the pxeboot vmlinuz file checksum""" > + return self._info.get('pxeboot_vmlinuz_chksum', None) > + > @property > def checksum(self): > """Gets the cloud-image file checksum""" > diff --git a/tests/acceptance/intel_iommu.py > b/tests/acceptance/intel_iommu.py > index bf8dea6e4f..a2f38ee2e9 100644 > --- a/tests/acceptance/intel_iommu.py > +++ b/tests/acceptance/intel_iommu.py > @@ -55,8 +55,10 @@ def common_vm_setup(self, custom_kernel=None): > > kernel_url = self.distro.pxeboot_url + 'vmlinuz' > initrd_url = self.distro.pxeboot_url + 'initrd.img' > - self.kernel_path = self.fetch_asset(kernel_url) > - self.initrd_path = self.fetch_asset(initrd_url) > + self.kernel_path = self.fetch_asset(kernel_url, > + asset_hash=self.distro.pxeboot_vmlinuz_chksum) > + self.initrd_path = self.fetch_asset(initrd_url, > + asset_hash=self.distro.pxeboot_initrd_chksum) > > def run_and_check(self): > if self.kernel_path: > >> >> Best Regards >> >> Eric >>> History: >>> v3 -> v4: >>> - I added Wainer's refactoring of KNOWN_DISTROS >>> into a class (last patch) and took into account his comments. >>> >>> v2 -> v3: >>> - Added Intel IOMMU tests were added. Different >>> operating modes are tested such as strict, caching mode, pt. >>> >>> Best Regards >>> >>> Eric >>> >>> The series and its dependencies can be found at: >>> https://github.com/eauger/qemu/tree/avocado-qemu-v4 >>> >>> Eric Auger (3): >>> Acceptance Tests: Add default kernel params and pxeboot url to the >>> KNOWN_DISTROS collection >>> avocado_qemu: Add SMMUv3 tests >>> avocado_qemu: Add Intel iommu tests >>> >>> Wainer dos Santos Moschetta (1): >>> avocado_qemu: Fix KNOWN_DISTROS map into the LinuxDistro class >>> >>> tests/acceptance/avocado_qemu/__init__.py | 118 +++++++++++++------ >>> tests/acceptance/intel_iommu.py | 115 +++++++++++++++++++ >>> tests/acceptance/smmu.py | 132 >>> ++++++++++++++++++++++ >>> 3 files changed, 332 insertions(+), 33 deletions(-) >>> create mode 100644 tests/acceptance/intel_iommu.py >>> create mode 100644 tests/acceptance/smmu.py >>> >
Hi Eric, On Mon, Jul 5, 2021 at 4:55 AM Eric Auger <eric.auger@redhat.com> wrote: > > Hi Wainer, > > On 7/1/21 1:22 AM, Wainer dos Santos Moschetta wrote: > > Hi, > > > > On 6/29/21 5:17 PM, Eric Auger wrote: > >> Hi Cleber, all, > >> > >> On 6/29/21 4:36 PM, Eric Auger wrote: > >>> This series adds ARM SMMU and Intel IOMMU functional > >>> tests using Fedora cloud-init images. > >>> > >>> ARM SMMU tests feature guests with and without RIL > >>> (range invalidation support) using respectively fedora 33 > >>> and 31. For each, we test the protection of virtio-net-pci > >>> and virtio-block-pci devices. Also strict=no and passthrough > >>> modes are tested. So there is a total of 6 tests. > >>> > >>> The series applies on top of Cleber's series: > >>> - [PATCH 0/3] Acceptance Tests: support choosing specific > >>> > >>> Note: > >>> - SMMU tests 2, 3, 5, 6 (resp. test_smmu_noril_passthrough and > >>> test_smmu_noril_nostrict) pass but the log reports: > >>> "WARN: Test passed but there were warnings during execution." > >>> This seems due to the lack of hash when fetching the kernel and > >>> initrd through fetch_asset(): > >>> WARNI| No hash provided. Cannot check the asset file integrity. > >> I wanted to emphasize that point and wondered how we could fix that > >> issue. Looks a pity the tests get tagged as WARN due to a lack of sha1. > >> Any advice? > > > > As Willian mentioned somewhere, to supress the WARN you can pass the > > kernel and initrd checksums (sha1) to the fetch_asset() method. > > > > Below is an draft implementation. It would need to fill out the > > remaining checksums and adjust the `smmu.py` tests. > > > > - Wainer > > > > ---- > > > > diff --git a/tests/acceptance/avocado_qemu/__init__.py > > b/tests/acceptance/avocado_qemu/__init__.py > > index 00eb0bfcc8..83637e2654 100644 > > --- a/tests/acceptance/avocado_qemu/__init__.py > > +++ b/tests/acceptance/avocado_qemu/__init__.py > > @@ -312,6 +312,8 @@ class LinuxDistro: > > {'checksum': > > 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0', > > 'pxeboot_url': > > "https://archives.fedoraproject.org/pub/archive/fedora/" > > "linux/releases/31/Everything/x86_64/os/images/pxeboot/", > > + 'pxeboot_initrd_chksum': > > 'dd0340a1b39bd28f88532babd4581c67649ec5b1', > > + 'pxeboot_vmlinuz_chksum': > > '5b6f6876e1b5bda314f93893271da0d5777b1f3c', > where did you get the checksum? I don't see any at the URL? Did you > generate it yourself? It is possible to use the hash you generate from the downloaded file. While I was reviewing this series, I thought it makes more sense to have Wainer's path applied first and then have your changes. I did this here, with the addition of myu suggestions in the series: https://gitlab.com/willianrampazzo/qemu/-/commits/test_eric_auger_v5. Feel free to pick it and resend a new version. Wainer, check if you agree with the changes to your patch and ack it. Regards, > > Thanks > > Eric > > 'kernel_params': > > "root=UUID=b1438b9b-2cab-4065-a99a-08a96687f73c ro " > > "no_timer_check net.ifnames=0 " > > "console=tty1 console=ttyS0,115200n8"}, > > @@ -371,6 +373,16 @@ def pxeboot_url(self): > > """Gets the repository url where pxeboot files can be found""" > > return self._info.get('pxeboot_url', None) > > > > + @property > > + def pxeboot_initrd_chksum(self): > > + """Gets the pxeboot initrd file checksum""" > > + return self._info.get('pxeboot_initrd_chksum', None) > > + > > + @property > > + def pxeboot_vmlinuz_chksum(self): > > + """Gets the pxeboot vmlinuz file checksum""" > > + return self._info.get('pxeboot_vmlinuz_chksum', None) > > + > > @property > > def checksum(self): > > """Gets the cloud-image file checksum""" > > diff --git a/tests/acceptance/intel_iommu.py > > b/tests/acceptance/intel_iommu.py > > index bf8dea6e4f..a2f38ee2e9 100644 > > --- a/tests/acceptance/intel_iommu.py > > +++ b/tests/acceptance/intel_iommu.py > > @@ -55,8 +55,10 @@ def common_vm_setup(self, custom_kernel=None): > > > > kernel_url = self.distro.pxeboot_url + 'vmlinuz' > > initrd_url = self.distro.pxeboot_url + 'initrd.img' > > - self.kernel_path = self.fetch_asset(kernel_url) > > - self.initrd_path = self.fetch_asset(initrd_url) > > + self.kernel_path = self.fetch_asset(kernel_url, > > + asset_hash=self.distro.pxeboot_vmlinuz_chksum) > > + self.initrd_path = self.fetch_asset(initrd_url, > > + asset_hash=self.distro.pxeboot_initrd_chksum) > > > > def run_and_check(self): > > if self.kernel_path: > > > >> > >> Best Regards > >> > >> Eric > >>> History: > >>> v3 -> v4: > >>> - I added Wainer's refactoring of KNOWN_DISTROS > >>> into a class (last patch) and took into account his comments. > >>> > >>> v2 -> v3: > >>> - Added Intel IOMMU tests were added. Different > >>> operating modes are tested such as strict, caching mode, pt. > >>> > >>> Best Regards > >>> > >>> Eric > >>> > >>> The series and its dependencies can be found at: > >>> https://github.com/eauger/qemu/tree/avocado-qemu-v4 > >>> > >>> Eric Auger (3): > >>> Acceptance Tests: Add default kernel params and pxeboot url to the > >>> KNOWN_DISTROS collection > >>> avocado_qemu: Add SMMUv3 tests > >>> avocado_qemu: Add Intel iommu tests > >>> > >>> Wainer dos Santos Moschetta (1): > >>> avocado_qemu: Fix KNOWN_DISTROS map into the LinuxDistro class > >>> > >>> tests/acceptance/avocado_qemu/__init__.py | 118 +++++++++++++------ > >>> tests/acceptance/intel_iommu.py | 115 +++++++++++++++++++ > >>> tests/acceptance/smmu.py | 132 > >>> ++++++++++++++++++++++ > >>> 3 files changed, 332 insertions(+), 33 deletions(-) > >>> create mode 100644 tests/acceptance/intel_iommu.py > >>> create mode 100644 tests/acceptance/smmu.py > >>> > > >
On 7/5/21 11:10 PM, Willian Rampazzo wrote: > Hi Eric, > > On Mon, Jul 5, 2021 at 4:55 AM Eric Auger <eric.auger@redhat.com> wrote: >> >> Hi Wainer, >> >> On 7/1/21 1:22 AM, Wainer dos Santos Moschetta wrote: >>> Hi, >>> >>> On 6/29/21 5:17 PM, Eric Auger wrote: >>>> Hi Cleber, all, >>>> >>>> On 6/29/21 4:36 PM, Eric Auger wrote: >>>>> This series adds ARM SMMU and Intel IOMMU functional >>>>> tests using Fedora cloud-init images. >>>>> >>>>> ARM SMMU tests feature guests with and without RIL >>>>> (range invalidation support) using respectively fedora 33 >>>>> and 31. For each, we test the protection of virtio-net-pci >>>>> and virtio-block-pci devices. Also strict=no and passthrough >>>>> modes are tested. So there is a total of 6 tests. >>>>> >>>>> The series applies on top of Cleber's series: >>>>> - [PATCH 0/3] Acceptance Tests: support choosing specific >>>>> >>>>> Note: >>>>> - SMMU tests 2, 3, 5, 6 (resp. test_smmu_noril_passthrough and >>>>> test_smmu_noril_nostrict) pass but the log reports: >>>>> "WARN: Test passed but there were warnings during execution." >>>>> This seems due to the lack of hash when fetching the kernel and >>>>> initrd through fetch_asset(): >>>>> WARNI| No hash provided. Cannot check the asset file integrity. >>>> I wanted to emphasize that point and wondered how we could fix that >>>> issue. Looks a pity the tests get tagged as WARN due to a lack of sha1. >>>> Any advice? >>> >>> As Willian mentioned somewhere, to supress the WARN you can pass the >>> kernel and initrd checksums (sha1) to the fetch_asset() method. >>> >>> Below is an draft implementation. It would need to fill out the >>> remaining checksums and adjust the `smmu.py` tests. >>> >>> - Wainer >>> >>> ---- >>> >>> diff --git a/tests/acceptance/avocado_qemu/__init__.py >>> b/tests/acceptance/avocado_qemu/__init__.py >>> index 00eb0bfcc8..83637e2654 100644 >>> --- a/tests/acceptance/avocado_qemu/__init__.py >>> +++ b/tests/acceptance/avocado_qemu/__init__.py >>> @@ -312,6 +312,8 @@ class LinuxDistro: >>> {'checksum': >>> 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0', >>> 'pxeboot_url': >>> "https://archives.fedoraproject.org/pub/archive/fedora/" >>> "linux/releases/31/Everything/x86_64/os/images/pxeboot/", >>> + 'pxeboot_initrd_chksum': >>> 'dd0340a1b39bd28f88532babd4581c67649ec5b1', >>> + 'pxeboot_vmlinuz_chksum': >>> '5b6f6876e1b5bda314f93893271da0d5777b1f3c', >> where did you get the checksum? I don't see any at the URL? Did you >> generate it yourself? > > It is possible to use the hash you generate from the downloaded file. > > While I was reviewing this series, I thought it makes more sense to > have Wainer's path applied first and then have your changes. I did > this here, with the addition of myu suggestions in the series: > https://gitlab.com/willianrampazzo/qemu/-/commits/test_eric_auger_v5. Off-list review is a bit unhandy (in particular when asked on the list). Why don't you post your improvements as v5? I don't think Eric will be offended: this is the opposite, you are helping him to get his patches merged ;) > Feel free to pick it and resend a new version. > > Wainer, check if you agree with the changes to your patch and ack it. > > Regards,
On Mon, Jul 5, 2021 at 6:20 PM Philippe Mathieu-Daudé <philmd@redhat.com> wrote: > > On 7/5/21 11:10 PM, Willian Rampazzo wrote: > > Hi Eric, > > > > On Mon, Jul 5, 2021 at 4:55 AM Eric Auger <eric.auger@redhat.com> wrote: > >> > >> Hi Wainer, > >> > >> On 7/1/21 1:22 AM, Wainer dos Santos Moschetta wrote: > >>> Hi, > >>> > >>> On 6/29/21 5:17 PM, Eric Auger wrote: > >>>> Hi Cleber, all, > >>>> > >>>> On 6/29/21 4:36 PM, Eric Auger wrote: > >>>>> This series adds ARM SMMU and Intel IOMMU functional > >>>>> tests using Fedora cloud-init images. > >>>>> > >>>>> ARM SMMU tests feature guests with and without RIL > >>>>> (range invalidation support) using respectively fedora 33 > >>>>> and 31. For each, we test the protection of virtio-net-pci > >>>>> and virtio-block-pci devices. Also strict=no and passthrough > >>>>> modes are tested. So there is a total of 6 tests. > >>>>> > >>>>> The series applies on top of Cleber's series: > >>>>> - [PATCH 0/3] Acceptance Tests: support choosing specific > >>>>> > >>>>> Note: > >>>>> - SMMU tests 2, 3, 5, 6 (resp. test_smmu_noril_passthrough and > >>>>> test_smmu_noril_nostrict) pass but the log reports: > >>>>> "WARN: Test passed but there were warnings during execution." > >>>>> This seems due to the lack of hash when fetching the kernel and > >>>>> initrd through fetch_asset(): > >>>>> WARNI| No hash provided. Cannot check the asset file integrity. > >>>> I wanted to emphasize that point and wondered how we could fix that > >>>> issue. Looks a pity the tests get tagged as WARN due to a lack of sha1. > >>>> Any advice? > >>> > >>> As Willian mentioned somewhere, to supress the WARN you can pass the > >>> kernel and initrd checksums (sha1) to the fetch_asset() method. > >>> > >>> Below is an draft implementation. It would need to fill out the > >>> remaining checksums and adjust the `smmu.py` tests. > >>> > >>> - Wainer > >>> > >>> ---- > >>> > >>> diff --git a/tests/acceptance/avocado_qemu/__init__.py > >>> b/tests/acceptance/avocado_qemu/__init__.py > >>> index 00eb0bfcc8..83637e2654 100644 > >>> --- a/tests/acceptance/avocado_qemu/__init__.py > >>> +++ b/tests/acceptance/avocado_qemu/__init__.py > >>> @@ -312,6 +312,8 @@ class LinuxDistro: > >>> {'checksum': > >>> 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0', > >>> 'pxeboot_url': > >>> "https://archives.fedoraproject.org/pub/archive/fedora/" > >>> "linux/releases/31/Everything/x86_64/os/images/pxeboot/", > >>> + 'pxeboot_initrd_chksum': > >>> 'dd0340a1b39bd28f88532babd4581c67649ec5b1', > >>> + 'pxeboot_vmlinuz_chksum': > >>> '5b6f6876e1b5bda314f93893271da0d5777b1f3c', > >> where did you get the checksum? I don't see any at the URL? Did you > >> generate it yourself? > > > > It is possible to use the hash you generate from the downloaded file. > > > > While I was reviewing this series, I thought it makes more sense to > > have Wainer's path applied first and then have your changes. I did > > this here, with the addition of myu suggestions in the series: > > https://gitlab.com/willianrampazzo/qemu/-/commits/test_eric_auger_v5. > > Off-list review is a bit unhandy (in particular when asked on the list). > > Why don't you post your improvements as v5? I don't think Eric will be > offended: this is the opposite, you are helping him to get his patches > merged ;) Oh, I did review each of his patches in the list and also already made the changes to speed up the process :) He mentioned today to me that his series is still depending on one from Cleber that was not merged yet, so we need to wait for that. > > > Feel free to pick it and resend a new version. > > > > Wainer, check if you agree with the changes to your patch and ack it. > > > > Regards, >
Hi William, Philippe, On 7/5/21 11:24 PM, Willian Rampazzo wrote: > On Mon, Jul 5, 2021 at 6:20 PM Philippe Mathieu-Daudé <philmd@redhat.com> wrote: >> On 7/5/21 11:10 PM, Willian Rampazzo wrote: >>> Hi Eric, >>> >>> On Mon, Jul 5, 2021 at 4:55 AM Eric Auger <eric.auger@redhat.com> wrote: >>>> Hi Wainer, >>>> >>>> On 7/1/21 1:22 AM, Wainer dos Santos Moschetta wrote: >>>>> Hi, >>>>> >>>>> On 6/29/21 5:17 PM, Eric Auger wrote: >>>>>> Hi Cleber, all, >>>>>> >>>>>> On 6/29/21 4:36 PM, Eric Auger wrote: >>>>>>> This series adds ARM SMMU and Intel IOMMU functional >>>>>>> tests using Fedora cloud-init images. >>>>>>> >>>>>>> ARM SMMU tests feature guests with and without RIL >>>>>>> (range invalidation support) using respectively fedora 33 >>>>>>> and 31. For each, we test the protection of virtio-net-pci >>>>>>> and virtio-block-pci devices. Also strict=no and passthrough >>>>>>> modes are tested. So there is a total of 6 tests. >>>>>>> >>>>>>> The series applies on top of Cleber's series: >>>>>>> - [PATCH 0/3] Acceptance Tests: support choosing specific >>>>>>> >>>>>>> Note: >>>>>>> - SMMU tests 2, 3, 5, 6 (resp. test_smmu_noril_passthrough and >>>>>>> test_smmu_noril_nostrict) pass but the log reports: >>>>>>> "WARN: Test passed but there were warnings during execution." >>>>>>> This seems due to the lack of hash when fetching the kernel and >>>>>>> initrd through fetch_asset(): >>>>>>> WARNI| No hash provided. Cannot check the asset file integrity. >>>>>> I wanted to emphasize that point and wondered how we could fix that >>>>>> issue. Looks a pity the tests get tagged as WARN due to a lack of sha1. >>>>>> Any advice? >>>>> As Willian mentioned somewhere, to supress the WARN you can pass the >>>>> kernel and initrd checksums (sha1) to the fetch_asset() method. >>>>> >>>>> Below is an draft implementation. It would need to fill out the >>>>> remaining checksums and adjust the `smmu.py` tests. >>>>> >>>>> - Wainer >>>>> >>>>> ---- >>>>> >>>>> diff --git a/tests/acceptance/avocado_qemu/__init__.py >>>>> b/tests/acceptance/avocado_qemu/__init__.py >>>>> index 00eb0bfcc8..83637e2654 100644 >>>>> --- a/tests/acceptance/avocado_qemu/__init__.py >>>>> +++ b/tests/acceptance/avocado_qemu/__init__.py >>>>> @@ -312,6 +312,8 @@ class LinuxDistro: >>>>> {'checksum': >>>>> 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0', >>>>> 'pxeboot_url': >>>>> "https://archives.fedoraproject.org/pub/archive/fedora/" >>>>> "linux/releases/31/Everything/x86_64/os/images/pxeboot/", >>>>> + 'pxeboot_initrd_chksum': >>>>> 'dd0340a1b39bd28f88532babd4581c67649ec5b1', >>>>> + 'pxeboot_vmlinuz_chksum': >>>>> '5b6f6876e1b5bda314f93893271da0d5777b1f3c', >>>> where did you get the checksum? I don't see any at the URL? Did you >>>> generate it yourself? >>> It is possible to use the hash you generate from the downloaded file. >>> >>> While I was reviewing this series, I thought it makes more sense to >>> have Wainer's path applied first and then have your changes. I did >>> this here, with the addition of myu suggestions in the series: >>> https://gitlab.com/willianrampazzo/qemu/-/commits/test_eric_auger_v5. >> Off-list review is a bit unhandy (in particular when asked on the list). >> >> Why don't you post your improvements as v5? I don't think Eric will be >> offended: this is the opposite, you are helping him to get his patches >> merged ;) > Oh, I did review each of his patches in the list and also already made > the changes to speed up the process :) Yes the review is really helpful. I will have a double check at your changes and respin quickly. > > He mentioned today to me that his series is still depending on one > from Cleber that was not merged yet, so we need to wait for that. Yep. The soft freeze is quickly approaching and that would be cool to see those tests landing upstream. Thanks Eric > >>> Feel free to pick it and resend a new version. >>> >>> Wainer, check if you agree with the changes to your patch and ack it. >>> >>> Regards,
This series adds ARM SMMU and Intel IOMMU functional tests using Fedora cloud-init images. ARM SMMU tests feature guests with and without RIL (range invalidation support) using respectively fedora 33 and 31. For each, we test the protection of virtio-net-pci and virtio-block-pci devices. Also strict=no and passthrough modes are tested. So there is a total of 6 tests. The series applies on top of Cleber's series: - [PATCH 0/3] Acceptance Tests: support choosing specific Note: - SMMU tests 2, 3, 5, 6 (resp. test_smmu_noril_passthrough and test_smmu_noril_nostrict) pass but the log reports: "WARN: Test passed but there were warnings during execution." This seems due to the lack of hash when fetching the kernel and initrd through fetch_asset(): WARNI| No hash provided. Cannot check the asset file integrity. History: v3 -> v4: - I added Wainer's refactoring of KNOWN_DISTROS into a class (last patch) and took into account his comments. v2 -> v3: - Added Intel IOMMU tests were added. Different operating modes are tested such as strict, caching mode, pt. Best Regards Eric The series and its dependencies can be found at: https://github.com/eauger/qemu/tree/avocado-qemu-v4 Eric Auger (3): Acceptance Tests: Add default kernel params and pxeboot url to the KNOWN_DISTROS collection avocado_qemu: Add SMMUv3 tests avocado_qemu: Add Intel iommu tests Wainer dos Santos Moschetta (1): avocado_qemu: Fix KNOWN_DISTROS map into the LinuxDistro class tests/acceptance/avocado_qemu/__init__.py | 118 +++++++++++++------ tests/acceptance/intel_iommu.py | 115 +++++++++++++++++++ tests/acceptance/smmu.py | 132 ++++++++++++++++++++++ 3 files changed, 332 insertions(+), 33 deletions(-) create mode 100644 tests/acceptance/intel_iommu.py create mode 100644 tests/acceptance/smmu.py