mbox

[PULL,00/32] Misc HW patches for 2025-02-10

Message ID 20250210204204.54407-1-philmd@linaro.org (mailing list archive)
State New
Headers show

Pull-request

https://github.com/philmd/qemu.git tags/hw-misc-20250210

Message

Philippe Mathieu-Daudé Feb. 10, 2025, 8:41 p.m. UTC
The following changes since commit 54e91d1523b412b4cff7cb36c898fa9dc133e886:

  Merge tag 'pull-qapi-2025-02-10-v2' of https://repo.or.cz/qemu/armbru into staging (2025-02-10 10:47:31 -0500)

are available in the Git repository at:

  https://github.com/philmd/qemu.git tags/hw-misc-20250210

for you to fetch changes up to 1078a376932cc1d1c501ee3643fef329da6a189a:

  hw/net/smc91c111: Ignore attempt to pop from empty RX fifo (2025-02-10 21:30:44 +0100)

----------------------------------------------------------------
Misc HW patches

- Use qemu_hexdump_line() in TPM backend (Philippe)
- Make various Xilinx devices endianness configurable (Philippe)
- Remove magic number in APIC (Phil)
- Disable thread-level cache topology (Zhao)
- Xen QOM style cleanups (Bernhard)
- Introduce TYPE_DYNAMIC_SYS_BUS_DEVICE (Philippe)
- Invert logic of machine no_sdcard flag (Philippe)
- Housekeeping in MicroBlaze functional tests (Philippe)
- Prevent out-of-bound access in SMC91C111 RX path (Peter)

----------------------------------------------------------------

Bernhard Beschow (1):
  hw/xen: Prefer QOM cast for XenLegacyDevice

Peter Maydell (1):
  hw/net/smc91c111: Ignore attempt to pop from empty RX fifo

Phil Dennis-Jordan (1):
  hw/intc/apic: Fixes magic number use, removes outdated comment

Philippe Mathieu-Daudé (28):
  backends/tpm: Use qemu_hexdump_line() to avoid sprintf()
  hw/intc/xilinx_intc: Make device endianness configurable
  hw/net/xilinx_ethlite: Make device endianness configurable
  hw/timer/xilinx_timer: Make device endianness configurable
  hw/char/xilinx_uartlite: Make device endianness configurable
  hw/ssi/xilinx_spi: Make device endianness configurable
  hw/arm/xlnx-zynqmp: Use &error_abort for programming errors
  hw/sysbus: Use sizeof(BusState) in main_system_bus_create()
  hw/sysbus: Declare QOM types using DEFINE_TYPES() macro
  hw/sysbus: Introduce TYPE_DYNAMIC_SYS_BUS_DEVICE
  hw/vfio: Have VFIO_PLATFORM devices inherit from
    DYNAMIC_SYS_BUS_DEVICE
  hw/display: Have RAMFB device inherit from DYNAMIC_SYS_BUS_DEVICE
  hw/i386: Have X86_IOMMU devices inherit from DYNAMIC_SYS_BUS_DEVICE
  hw/net: Have eTSEC device inherit from DYNAMIC_SYS_BUS_DEVICE
  hw/tpm: Have TPM TIS sysbus device inherit from DYNAMIC_SYS_BUS_DEVICE
  hw/xen: Have legacy Xen backend inherit from DYNAMIC_SYS_BUS_DEVICE
  hw/boards: Convert no_sdcard flag to OnOffAuto tri-state
  hw/boards: Explicit no_sdcard=false as ON_OFF_AUTO_OFF
  hw/boards: Rename no_sdcard -> auto_create_sdcard
  hw/boards: Do not create unusable default if=sd drives
  hw/arm: Remove all invalid uses of auto_create_sdcard=true
  hw/riscv: Remove all invalid uses of auto_create_sdcard=true
  hw/boards: Ensure machine setting auto_create_sdcard expose a SD Bus
  tests/functional: Explicit endianness of microblaze assets
  tests/functional: Allow microblaze tests to take a machine name
    argument
  tests/functional: Remove sleep() kludges from microblaze tests
  tests/functional: Have microblaze tests inherit common parent class
  hw/riscv/opentitan: Include missing 'exec/address-spaces.h' header

Zhao Liu (1):
  hw/core/machine: Reject thread level cache

 include/hw/boards.h                           |  2 +-
 include/hw/sysbus.h                           |  2 +
 include/hw/xen/xen_pvdev.h                    |  5 +-
 backends/tpm/tpm_util.c                       | 24 ++++----
 hw/arm/aspeed.c                               | 20 +++++++
 hw/arm/bananapi_m2u.c                         |  1 +
 hw/arm/cubieboard.c                           |  1 +
 hw/arm/exynos4_boards.c                       |  2 +
 hw/arm/fby35.c                                |  1 +
 hw/arm/imx25_pdk.c                            |  1 +
 hw/arm/integratorcp.c                         |  1 +
 hw/arm/mcimx6ul-evk.c                         |  1 +
 hw/arm/mcimx7d-sabre.c                        |  1 +
 hw/arm/npcm7xx_boards.c                       |  5 ++
 hw/arm/omap_sx1.c                             |  2 +
 hw/arm/orangepi.c                             |  1 +
 hw/arm/raspi.c                                |  5 ++
 hw/arm/raspi4b.c                              |  1 +
 hw/arm/realview.c                             |  4 ++
 hw/arm/sabrelite.c                            |  1 +
 hw/arm/stellaris.c                            |  1 +
 hw/arm/versatilepb.c                          |  2 +
 hw/arm/vexpress.c                             |  2 +
 hw/arm/xilinx_zynq.c                          |  1 -
 hw/arm/xlnx-versal-virt.c                     |  1 +
 hw/arm/xlnx-zcu102.c                          |  1 +
 hw/arm/xlnx-zynqmp.c                          | 38 ++++--------
 hw/char/xilinx_uartlite.c                     | 27 +++++----
 hw/core/machine-smp.c                         |  7 +++
 hw/core/null-machine.c                        |  1 -
 hw/core/sysbus.c                              | 54 ++++++++++-------
 hw/display/ramfb-standalone.c                 |  3 +-
 hw/i386/amd_iommu.c                           |  2 -
 hw/i386/intel_iommu.c                         |  2 -
 hw/i386/x86-iommu.c                           |  2 +-
 hw/intc/apic.c                                |  3 +-
 hw/intc/xilinx_intc.c                         | 52 +++++++++++-----
 hw/microblaze/petalogix_ml605_mmu.c           |  3 +
 hw/microblaze/petalogix_s3adsp1800_mmu.c      |  4 ++
 hw/net/fsl_etsec/etsec.c                      |  4 +-
 hw/net/smc91c111.c                            |  9 +++
 hw/net/xilinx_ethlite.c                       | 20 +++++--
 hw/ppc/virtex_ml507.c                         |  1 +
 hw/riscv/microchip_pfsoc.c                    |  1 +
 hw/riscv/opentitan.c                          |  1 +
 hw/riscv/sifive_u.c                           |  1 +
 hw/s390x/s390-virtio-ccw.c                    |  1 -
 hw/ssi/xilinx_spi.c                           | 24 +++++---
 hw/timer/xilinx_timer.c                       | 35 +++++++----
 hw/tpm/tpm_tis_sysbus.c                       |  3 +-
 hw/usb/xen-usb.c                              |  6 +-
 hw/vfio/amd-xgbe.c                            |  2 -
 hw/vfio/calxeda-xgmac.c                       |  2 -
 hw/vfio/platform.c                            |  4 +-
 hw/xen/xen-legacy-backend.c                   |  9 +--
 hw/xen/xen_pvdev.c                            |  2 +-
 system/vl.c                                   | 24 ++++++--
 .../functional/test_microblaze_s3adsp1800.py  | 34 +++++++++--
 .../test_microblazeel_s3adsp1800.py           | 32 ++--------
 tests/qemu-iotests/172.out                    | 60 -------------------
 60 files changed, 314 insertions(+), 248 deletions(-)

Comments

Stefan Hajnoczi Feb. 11, 2025, 6:26 p.m. UTC | #1
On Mon, Feb 10, 2025 at 3:43 PM Philippe Mathieu-Daudé
<philmd@linaro.org> wrote:
>
> The following changes since commit 54e91d1523b412b4cff7cb36c898fa9dc133e886:
>
>   Merge tag 'pull-qapi-2025-02-10-v2' of https://repo.or.cz/qemu/armbru into staging (2025-02-10 10:47:31 -0500)
>
> are available in the Git repository at:
>
>   https://github.com/philmd/qemu.git tags/hw-misc-20250210
>
> for you to fetch changes up to 1078a376932cc1d1c501ee3643fef329da6a189a:
>
>   hw/net/smc91c111: Ignore attempt to pop from empty RX fifo (2025-02-10 21:30:44 +0100)
>
> ----------------------------------------------------------------
> Misc HW patches
>
> - Use qemu_hexdump_line() in TPM backend (Philippe)
> - Make various Xilinx devices endianness configurable (Philippe)
> - Remove magic number in APIC (Phil)
> - Disable thread-level cache topology (Zhao)
> - Xen QOM style cleanups (Bernhard)
> - Introduce TYPE_DYNAMIC_SYS_BUS_DEVICE (Philippe)
> - Invert logic of machine no_sdcard flag (Philippe)
> - Housekeeping in MicroBlaze functional tests (Philippe)

Please take a look at this CI failure:
https://gitlab.com/qemu-project/qemu/-/jobs/9106591368

Thanks,
Stefan

> - Prevent out-of-bound access in SMC91C111 RX path (Peter)
>
> ----------------------------------------------------------------
>
> Bernhard Beschow (1):
>   hw/xen: Prefer QOM cast for XenLegacyDevice
>
> Peter Maydell (1):
>   hw/net/smc91c111: Ignore attempt to pop from empty RX fifo
>
> Phil Dennis-Jordan (1):
>   hw/intc/apic: Fixes magic number use, removes outdated comment
>
> Philippe Mathieu-Daudé (28):
>   backends/tpm: Use qemu_hexdump_line() to avoid sprintf()
>   hw/intc/xilinx_intc: Make device endianness configurable
>   hw/net/xilinx_ethlite: Make device endianness configurable
>   hw/timer/xilinx_timer: Make device endianness configurable
>   hw/char/xilinx_uartlite: Make device endianness configurable
>   hw/ssi/xilinx_spi: Make device endianness configurable
>   hw/arm/xlnx-zynqmp: Use &error_abort for programming errors
>   hw/sysbus: Use sizeof(BusState) in main_system_bus_create()
>   hw/sysbus: Declare QOM types using DEFINE_TYPES() macro
>   hw/sysbus: Introduce TYPE_DYNAMIC_SYS_BUS_DEVICE
>   hw/vfio: Have VFIO_PLATFORM devices inherit from
>     DYNAMIC_SYS_BUS_DEVICE
>   hw/display: Have RAMFB device inherit from DYNAMIC_SYS_BUS_DEVICE
>   hw/i386: Have X86_IOMMU devices inherit from DYNAMIC_SYS_BUS_DEVICE
>   hw/net: Have eTSEC device inherit from DYNAMIC_SYS_BUS_DEVICE
>   hw/tpm: Have TPM TIS sysbus device inherit from DYNAMIC_SYS_BUS_DEVICE
>   hw/xen: Have legacy Xen backend inherit from DYNAMIC_SYS_BUS_DEVICE
>   hw/boards: Convert no_sdcard flag to OnOffAuto tri-state
>   hw/boards: Explicit no_sdcard=false as ON_OFF_AUTO_OFF
>   hw/boards: Rename no_sdcard -> auto_create_sdcard
>   hw/boards: Do not create unusable default if=sd drives
>   hw/arm: Remove all invalid uses of auto_create_sdcard=true
>   hw/riscv: Remove all invalid uses of auto_create_sdcard=true
>   hw/boards: Ensure machine setting auto_create_sdcard expose a SD Bus
>   tests/functional: Explicit endianness of microblaze assets
>   tests/functional: Allow microblaze tests to take a machine name
>     argument
>   tests/functional: Remove sleep() kludges from microblaze tests
>   tests/functional: Have microblaze tests inherit common parent class
>   hw/riscv/opentitan: Include missing 'exec/address-spaces.h' header
>
> Zhao Liu (1):
>   hw/core/machine: Reject thread level cache
>
>  include/hw/boards.h                           |  2 +-
>  include/hw/sysbus.h                           |  2 +
>  include/hw/xen/xen_pvdev.h                    |  5 +-
>  backends/tpm/tpm_util.c                       | 24 ++++----
>  hw/arm/aspeed.c                               | 20 +++++++
>  hw/arm/bananapi_m2u.c                         |  1 +
>  hw/arm/cubieboard.c                           |  1 +
>  hw/arm/exynos4_boards.c                       |  2 +
>  hw/arm/fby35.c                                |  1 +
>  hw/arm/imx25_pdk.c                            |  1 +
>  hw/arm/integratorcp.c                         |  1 +
>  hw/arm/mcimx6ul-evk.c                         |  1 +
>  hw/arm/mcimx7d-sabre.c                        |  1 +
>  hw/arm/npcm7xx_boards.c                       |  5 ++
>  hw/arm/omap_sx1.c                             |  2 +
>  hw/arm/orangepi.c                             |  1 +
>  hw/arm/raspi.c                                |  5 ++
>  hw/arm/raspi4b.c                              |  1 +
>  hw/arm/realview.c                             |  4 ++
>  hw/arm/sabrelite.c                            |  1 +
>  hw/arm/stellaris.c                            |  1 +
>  hw/arm/versatilepb.c                          |  2 +
>  hw/arm/vexpress.c                             |  2 +
>  hw/arm/xilinx_zynq.c                          |  1 -
>  hw/arm/xlnx-versal-virt.c                     |  1 +
>  hw/arm/xlnx-zcu102.c                          |  1 +
>  hw/arm/xlnx-zynqmp.c                          | 38 ++++--------
>  hw/char/xilinx_uartlite.c                     | 27 +++++----
>  hw/core/machine-smp.c                         |  7 +++
>  hw/core/null-machine.c                        |  1 -
>  hw/core/sysbus.c                              | 54 ++++++++++-------
>  hw/display/ramfb-standalone.c                 |  3 +-
>  hw/i386/amd_iommu.c                           |  2 -
>  hw/i386/intel_iommu.c                         |  2 -
>  hw/i386/x86-iommu.c                           |  2 +-
>  hw/intc/apic.c                                |  3 +-
>  hw/intc/xilinx_intc.c                         | 52 +++++++++++-----
>  hw/microblaze/petalogix_ml605_mmu.c           |  3 +
>  hw/microblaze/petalogix_s3adsp1800_mmu.c      |  4 ++
>  hw/net/fsl_etsec/etsec.c                      |  4 +-
>  hw/net/smc91c111.c                            |  9 +++
>  hw/net/xilinx_ethlite.c                       | 20 +++++--
>  hw/ppc/virtex_ml507.c                         |  1 +
>  hw/riscv/microchip_pfsoc.c                    |  1 +
>  hw/riscv/opentitan.c                          |  1 +
>  hw/riscv/sifive_u.c                           |  1 +
>  hw/s390x/s390-virtio-ccw.c                    |  1 -
>  hw/ssi/xilinx_spi.c                           | 24 +++++---
>  hw/timer/xilinx_timer.c                       | 35 +++++++----
>  hw/tpm/tpm_tis_sysbus.c                       |  3 +-
>  hw/usb/xen-usb.c                              |  6 +-
>  hw/vfio/amd-xgbe.c                            |  2 -
>  hw/vfio/calxeda-xgmac.c                       |  2 -
>  hw/vfio/platform.c                            |  4 +-
>  hw/xen/xen-legacy-backend.c                   |  9 +--
>  hw/xen/xen_pvdev.c                            |  2 +-
>  system/vl.c                                   | 24 ++++++--
>  .../functional/test_microblaze_s3adsp1800.py  | 34 +++++++++--
>  .../test_microblazeel_s3adsp1800.py           | 32 ++--------
>  tests/qemu-iotests/172.out                    | 60 -------------------
>  60 files changed, 314 insertions(+), 248 deletions(-)
>
> --
> 2.47.1
>
>
Philippe Mathieu-Daudé Feb. 11, 2025, 6:48 p.m. UTC | #2
On 11/2/25 19:26, Stefan Hajnoczi wrote:
> On Mon, Feb 10, 2025 at 3:43 PM Philippe Mathieu-Daudé
> <philmd@linaro.org> wrote:
>>
>> The following changes since commit 54e91d1523b412b4cff7cb36c898fa9dc133e886:
>>
>>    Merge tag 'pull-qapi-2025-02-10-v2' of https://repo.or.cz/qemu/armbru into staging (2025-02-10 10:47:31 -0500)
>>
>> are available in the Git repository at:
>>
>>    https://github.com/philmd/qemu.git tags/hw-misc-20250210
>>
>> for you to fetch changes up to 1078a376932cc1d1c501ee3643fef329da6a189a:
>>
>>    hw/net/smc91c111: Ignore attempt to pop from empty RX fifo (2025-02-10 21:30:44 +0100)
>>
>> ----------------------------------------------------------------
>> Misc HW patches
>>
>> - Use qemu_hexdump_line() in TPM backend (Philippe)
>> - Make various Xilinx devices endianness configurable (Philippe)
>> - Remove magic number in APIC (Phil)
>> - Disable thread-level cache topology (Zhao)
>> - Xen QOM style cleanups (Bernhard)
>> - Introduce TYPE_DYNAMIC_SYS_BUS_DEVICE (Philippe)
>> - Invert logic of machine no_sdcard flag (Philippe)
>> - Housekeeping in MicroBlaze functional tests (Philippe)
> 
> Please take a look at this CI failure:
> https://gitlab.com/qemu-project/qemu/-/jobs/9106591368

Hmm I can not reproduce locally this error:

   Exception: Asset cache is invalid and downloads disabled

https://gitlab.com/qemu-project/qemu/-/jobs/9106591368/artifacts/external_file/build/tests/functional/microblazeel/test_microblazeel_s3adsp1800.MicroblazeLittleEndianMachine.test_microblaze_s3adsp1800_legacy_le/base.log 
content is:

   2025-02-11 16:24:55,525 - DEBUG: Extract 
/builds/qemu-project/qemu/functional-cache/download/b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22 
format=Nonesub_dir=None member=None

which is correct:

$ sha256sum day13.tar.gz
b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22 
day13.tar.gz

Did you restart the job to see if it is a network issue?

Regards,

Phil.
Philippe Mathieu-Daudé Feb. 11, 2025, 6:53 p.m. UTC | #3
On 11/2/25 19:48, Philippe Mathieu-Daudé wrote:
> On 11/2/25 19:26, Stefan Hajnoczi wrote:
>> On Mon, Feb 10, 2025 at 3:43 PM Philippe Mathieu-Daudé
>> <philmd@linaro.org> wrote:
>>>
>>> The following changes since commit 
>>> 54e91d1523b412b4cff7cb36c898fa9dc133e886:
>>>
>>>    Merge tag 'pull-qapi-2025-02-10-v2' of https://repo.or.cz/qemu/ 
>>> armbru into staging (2025-02-10 10:47:31 -0500)
>>>
>>> are available in the Git repository at:
>>>
>>>    https://github.com/philmd/qemu.git tags/hw-misc-20250210
>>>
>>> for you to fetch changes up to 1078a376932cc1d1c501ee3643fef329da6a189a:
>>>
>>>    hw/net/smc91c111: Ignore attempt to pop from empty RX fifo 
>>> (2025-02-10 21:30:44 +0100)
>>>
>>> ----------------------------------------------------------------
>>> Misc HW patches
>>>
>>> - Use qemu_hexdump_line() in TPM backend (Philippe)
>>> - Make various Xilinx devices endianness configurable (Philippe)
>>> - Remove magic number in APIC (Phil)
>>> - Disable thread-level cache topology (Zhao)
>>> - Xen QOM style cleanups (Bernhard)
>>> - Introduce TYPE_DYNAMIC_SYS_BUS_DEVICE (Philippe)
>>> - Invert logic of machine no_sdcard flag (Philippe)
>>> - Housekeeping in MicroBlaze functional tests (Philippe)
>>
>> Please take a look at this CI failure:
>> https://gitlab.com/qemu-project/qemu/-/jobs/9106591368
> 
> Hmm I can not reproduce locally this error:
> 
>    Exception: Asset cache is invalid and downloads disabled

OK, I could reproduce by blowing my cache away.

The problem seems in the "tests/functional: Have microblaze tests
inherit common parent class" patch, which does:

-class MicroblazeelMachine(QemuSystemTest):
+class MicroblazeLittleEndianMachine(MicroblazeMachine):

I presume, since MicroblazeLittleEndianMachine is no more a direct
child of QemuSystemTest, then the ASSET_IMAGE_* aren't automatically
downloaded.

> https://gitlab.com/qemu-project/qemu/-/jobs/9106591368/artifacts/ 
> external_file/build/tests/functional/microblazeel/ 
> test_microblazeel_s3adsp1800.MicroblazeLittleEndianMachine.test_microblaze_s3adsp1800_legacy_le/base.log content is:
> 
>    2025-02-11 16:24:55,525 - DEBUG: Extract /builds/qemu-project/qemu/ 
> functional-cache/download/ 
> b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22 
> format=Nonesub_dir=None member=None
> 
> which is correct:
> 
> $ sha256sum day13.tar.gz
> b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22 
> day13.tar.gz
> 
> Did you restart the job to see if it is a network issue?
> 
> Regards,
> 
> Phil.
Philippe Mathieu-Daudé Feb. 11, 2025, 7:03 p.m. UTC | #4
On 11/2/25 19:53, Philippe Mathieu-Daudé wrote:
> On 11/2/25 19:48, Philippe Mathieu-Daudé wrote:
>> On 11/2/25 19:26, Stefan Hajnoczi wrote:
>>> On Mon, Feb 10, 2025 at 3:43 PM Philippe Mathieu-Daudé
>>> <philmd@linaro.org> wrote:
>>>>
>>>> The following changes since commit 
>>>> 54e91d1523b412b4cff7cb36c898fa9dc133e886:
>>>>
>>>>    Merge tag 'pull-qapi-2025-02-10-v2' of https://repo.or.cz/qemu/ 
>>>> armbru into staging (2025-02-10 10:47:31 -0500)
>>>>
>>>> are available in the Git repository at:
>>>>
>>>>    https://github.com/philmd/qemu.git tags/hw-misc-20250210
>>>>
>>>> for you to fetch changes up to 
>>>> 1078a376932cc1d1c501ee3643fef329da6a189a:
>>>>
>>>>    hw/net/smc91c111: Ignore attempt to pop from empty RX fifo 
>>>> (2025-02-10 21:30:44 +0100)
>>>>
>>>> ----------------------------------------------------------------
>>>> Misc HW patches
>>>>
>>>> - Use qemu_hexdump_line() in TPM backend (Philippe)
>>>> - Make various Xilinx devices endianness configurable (Philippe)
>>>> - Remove magic number in APIC (Phil)
>>>> - Disable thread-level cache topology (Zhao)
>>>> - Xen QOM style cleanups (Bernhard)
>>>> - Introduce TYPE_DYNAMIC_SYS_BUS_DEVICE (Philippe)
>>>> - Invert logic of machine no_sdcard flag (Philippe)
>>>> - Housekeeping in MicroBlaze functional tests (Philippe)
>>>
>>> Please take a look at this CI failure:
>>> https://gitlab.com/qemu-project/qemu/-/jobs/9106591368
>>
>> Hmm I can not reproduce locally this error:
>>
>>    Exception: Asset cache is invalid and downloads disabled
> 
> OK, I could reproduce by blowing my cache away.
> 
> The problem seems in the "tests/functional: Have microblaze tests
> inherit common parent class" patch, which does:
> 
> -class MicroblazeelMachine(QemuSystemTest):
> +class MicroblazeLittleEndianMachine(MicroblazeMachine):
> 
> I presume, since MicroblazeLittleEndianMachine is no more a direct
> child of QemuSystemTest, then the ASSET_IMAGE_* aren't automatically
> downloaded.

Well, apparently related to network failure:

INFO:qemu-test:Downloading 
http://www.qemu-advent-calendar.org/2023/download/day13.tar.gz to 
/Users/philmd/.cache/qemu/download/b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22...
ERROR:qemu-test:Unable to download 
http://www.qemu-advent-calendar.org/2023/download/day13.tar.gz: HTTP 
Error 403: Forbidden

$ curl -v http://www.qemu-advent-calendar.org/2023/download/day13.tar.gz
 > GET /2023/download/day13.tar.gz HTTP/1.1
< HTTP/1.1 301 Moved Permanently
< Location: https://www.qemu-advent-calendar.org/2023/download/day13.tar.gz

Using:

-- >8 --
diff --git a/tests/functional/test_microblaze_s3adsp1800.py 
b/tests/functional/test_microblaze_s3adsp1800.py
index 177c8a685bc..949e627c84a 100755
--- a/tests/functional/test_microblaze_s3adsp1800.py
+++ b/tests/functional/test_microblaze_s3adsp1800.py
@@ -24,3 +24,3 @@ class MicroblazeMachine(QemuSystemTest):
      ASSET_IMAGE_LE = Asset(
-        ('http://www.qemu-advent-calendar.org/2023/download/day13.tar.gz'),
+ 
('https://www.qemu-advent-calendar.org/2023/download/day13.tar.gz'),
  
'b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22')
---

I still get:

INFO:qemu-test:Downloading 
https://www.qemu-advent-calendar.org/2023/download/day13.tar.gz to 
/Users/philmd/.cache/qemu/download/b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22...

However:

$ curl --http1.0 -v 
https://www.qemu-advent-calendar.org/2023/download/day13.tar.gz
 > GET /2023/download/day13.tar.gz HTTP/1.0
< HTTP/1.0 200 OK
< Content-Length: 4752277
< Content-Type: application/gzip

So I'm confused...

> 
>> https://gitlab.com/qemu-project/qemu/-/jobs/9106591368/artifacts/ 
>> external_file/build/tests/functional/microblazeel/ 
>> test_microblazeel_s3adsp1800.MicroblazeLittleEndianMachine.test_microblaze_s3adsp1800_legacy_le/base.log content is:
>>
>>    2025-02-11 16:24:55,525 - DEBUG: Extract /builds/qemu-project/qemu/ 
>> functional-cache/download/ 
>> b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22 
>> format=Nonesub_dir=None member=None
>>
>> which is correct:
>>
>> $ sha256sum day13.tar.gz
>> b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22 
>> day13.tar.gz
>>
>> Did you restart the job to see if it is a network issue?
>>
>> Regards,
>>
>> Phil.
>
Thomas Huth Feb. 12, 2025, 12:31 a.m. UTC | #5
On 11/02/2025 19.53, Philippe Mathieu-Daudé wrote:
> On 11/2/25 19:48, Philippe Mathieu-Daudé wrote:
>> On 11/2/25 19:26, Stefan Hajnoczi wrote:
>>> On Mon, Feb 10, 2025 at 3:43 PM Philippe Mathieu-Daudé
>>> <philmd@linaro.org> wrote:
>>>>
>>>> The following changes since commit 
>>>> 54e91d1523b412b4cff7cb36c898fa9dc133e886:
>>>>
>>>>    Merge tag 'pull-qapi-2025-02-10-v2' of https://repo.or.cz/qemu/ 
>>>> armbru into staging (2025-02-10 10:47:31 -0500)
>>>>
>>>> are available in the Git repository at:
>>>>
>>>>    https://github.com/philmd/qemu.git tags/hw-misc-20250210
>>>>
>>>> for you to fetch changes up to 1078a376932cc1d1c501ee3643fef329da6a189a:
>>>>
>>>>    hw/net/smc91c111: Ignore attempt to pop from empty RX fifo 
>>>> (2025-02-10 21:30:44 +0100)
>>>>
>>>> ----------------------------------------------------------------
>>>> Misc HW patches
>>>>
>>>> - Use qemu_hexdump_line() in TPM backend (Philippe)
>>>> - Make various Xilinx devices endianness configurable (Philippe)
>>>> - Remove magic number in APIC (Phil)
>>>> - Disable thread-level cache topology (Zhao)
>>>> - Xen QOM style cleanups (Bernhard)
>>>> - Introduce TYPE_DYNAMIC_SYS_BUS_DEVICE (Philippe)
>>>> - Invert logic of machine no_sdcard flag (Philippe)
>>>> - Housekeeping in MicroBlaze functional tests (Philippe)
>>>
>>> Please take a look at this CI failure:
>>> https://gitlab.com/qemu-project/qemu/-/jobs/9106591368
>>
>> Hmm I can not reproduce locally this error:
>>
>>    Exception: Asset cache is invalid and downloads disabled
> 
> OK, I could reproduce by blowing my cache away.
> 
> The problem seems in the "tests/functional: Have microblaze tests
> inherit common parent class" patch, which does:
> 
> -class MicroblazeelMachine(QemuSystemTest):
> +class MicroblazeLittleEndianMachine(MicroblazeMachine):
> 
> I presume, since MicroblazeLittleEndianMachine is no more a direct
> child of QemuSystemTest, then the ASSET_IMAGE_* aren't automatically
> downloaded.

I guess you could fix it with:

diff --git a/tests/functional/test_microblaze_s3adsp1800.py 
b/tests/functional/test_microblaze_s3adsp1800.py
--- a/tests/functional/test_microblaze_s3adsp1800.py
+++ b/tests/functional/test_microblaze_s3adsp1800.py
@@ -57,6 +57,8 @@ def do_xmaton_le_test(self, machine):

  class MicroblazeBigEndianMachine(MicroblazeMachine):

+    ASSET_IMAGE_BE = MicroblazeMachine.ASSET_IMAGE_BE
+
      def test_microblaze_s3adsp1800_legacy_be(self):
          self.do_ballerina_be_test('petalogix-s3adsp1800')

diff --git a/tests/functional/test_microblazeel_s3adsp1800.py 
b/tests/functional/test_microblazeel_s3adsp1800.py
--- a/tests/functional/test_microblazeel_s3adsp1800.py
+++ b/tests/functional/test_microblazeel_s3adsp1800.py
@@ -13,6 +13,8 @@

  class MicroblazeLittleEndianMachine(MicroblazeMachine):

+    ASSET_IMAGE_LE = MicroblazeMachine.ASSET_IMAGE_LE
+
      def test_microblaze_s3adsp1800_legacy_le(self):
          self.do_xmaton_le_test('petalogix-s3adsp1800')

(this still won't help with the strange 403 error ... maybe we have to 
mirror the asset in another place now if the advent-calendar site does not 
work reliably anymore...)

  Thomas
Thomas Huth Feb. 12, 2025, 12:41 a.m. UTC | #6
On 11/02/2025 20.03, Philippe Mathieu-Daudé wrote:
> On 11/2/25 19:53, Philippe Mathieu-Daudé wrote:
>> On 11/2/25 19:48, Philippe Mathieu-Daudé wrote:
>>> On 11/2/25 19:26, Stefan Hajnoczi wrote:
>>>> On Mon, Feb 10, 2025 at 3:43 PM Philippe Mathieu-Daudé
>>>> <philmd@linaro.org> wrote:
>>>>>
>>>>> The following changes since commit 
>>>>> 54e91d1523b412b4cff7cb36c898fa9dc133e886:
>>>>>
>>>>>    Merge tag 'pull-qapi-2025-02-10-v2' of https://repo.or.cz/qemu/ 
>>>>> armbru into staging (2025-02-10 10:47:31 -0500)
>>>>>
>>>>> are available in the Git repository at:
>>>>>
>>>>>    https://github.com/philmd/qemu.git tags/hw-misc-20250210
>>>>>
>>>>> for you to fetch changes up to 1078a376932cc1d1c501ee3643fef329da6a189a:
>>>>>
>>>>>    hw/net/smc91c111: Ignore attempt to pop from empty RX fifo 
>>>>> (2025-02-10 21:30:44 +0100)
>>>>>
>>>>> ----------------------------------------------------------------
>>>>> Misc HW patches
>>>>>
>>>>> - Use qemu_hexdump_line() in TPM backend (Philippe)
>>>>> - Make various Xilinx devices endianness configurable (Philippe)
>>>>> - Remove magic number in APIC (Phil)
>>>>> - Disable thread-level cache topology (Zhao)
>>>>> - Xen QOM style cleanups (Bernhard)
>>>>> - Introduce TYPE_DYNAMIC_SYS_BUS_DEVICE (Philippe)
>>>>> - Invert logic of machine no_sdcard flag (Philippe)
>>>>> - Housekeeping in MicroBlaze functional tests (Philippe)
>>>>
>>>> Please take a look at this CI failure:
>>>> https://gitlab.com/qemu-project/qemu/-/jobs/9106591368
>>>
>>> Hmm I can not reproduce locally this error:
>>>
>>>    Exception: Asset cache is invalid and downloads disabled
>>
>> OK, I could reproduce by blowing my cache away.
>>
>> The problem seems in the "tests/functional: Have microblaze tests
>> inherit common parent class" patch, which does:
>>
>> -class MicroblazeelMachine(QemuSystemTest):
>> +class MicroblazeLittleEndianMachine(MicroblazeMachine):
>>
>> I presume, since MicroblazeLittleEndianMachine is no more a direct
>> child of QemuSystemTest, then the ASSET_IMAGE_* aren't automatically
>> downloaded.
> 
> Well, apparently related to network failure:
> 
> INFO:qemu-test:Downloading http://www.qemu-advent-calendar.org/2023/ 
> download/day13.tar.gz to /Users/philmd/.cache/qemu/download/ 
> b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22...
> ERROR:qemu-test:Unable to download http://www.qemu-advent-calendar.org/2023/ 
> download/day13.tar.gz: HTTP Error 403: Forbidden
> 
> $ curl -v http://www.qemu-advent-calendar.org/2023/download/day13.tar.gz
>  > GET /2023/download/day13.tar.gz HTTP/1.1
> < HTTP/1.1 301 Moved Permanently
> < Location: https://www.qemu-advent-calendar.org/2023/download/day13.tar.gz
> 
> Using:
> 
> -- >8 --
> diff --git a/tests/functional/test_microblaze_s3adsp1800.py b/tests/ 
> functional/test_microblaze_s3adsp1800.py
> index 177c8a685bc..949e627c84a 100755
> --- a/tests/functional/test_microblaze_s3adsp1800.py
> +++ b/tests/functional/test_microblaze_s3adsp1800.py
> @@ -24,3 +24,3 @@ class MicroblazeMachine(QemuSystemTest):
>       ASSET_IMAGE_LE = Asset(
> -        ('http://www.qemu-advent-calendar.org/2023/download/day13.tar.gz'),
> + ('https://www.qemu-advent-calendar.org/2023/download/day13.tar.gz'),
> 
> 'b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22')
> ---
> 
> I still get:
> 
> INFO:qemu-test:Downloading https://www.qemu-advent-calendar.org/2023/ 
> download/day13.tar.gz to /Users/philmd/.cache/qemu/download/ 
> b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22...
> 
> However:
> 
> $ curl --http1.0 -v https://www.qemu-advent-calendar.org/2023/download/ 
> day13.tar.gz
>  > GET /2023/download/day13.tar.gz HTTP/1.0
> < HTTP/1.0 200 OK
> < Content-Length: 4752277
> < Content-Type: application/gzip
> 
> So I'm confused...

Looks like this also happens in test runs without your patches:

  https://gitlab.com/qemu-project/qemu/-/jobs/9108828196#L844

The test then silently gets skipped there (since we only fail hard for a 404 
error now).

Maybe Eldon could comment why the downloads are blocked for python scripts 
but not for curl downloads?

I the worst case, we have to mirror the asset to another place, I guess...

  Thomas
Thomas Huth Feb. 12, 2025, 1:39 a.m. UTC | #7
On 12/02/2025 01.41, Thomas Huth wrote:
> On 11/02/2025 20.03, Philippe Mathieu-Daudé wrote:
>> On 11/2/25 19:53, Philippe Mathieu-Daudé wrote:
>>> On 11/2/25 19:48, Philippe Mathieu-Daudé wrote:
>>>> On 11/2/25 19:26, Stefan Hajnoczi wrote:
>>>>> On Mon, Feb 10, 2025 at 3:43 PM Philippe Mathieu-Daudé
>>>>> <philmd@linaro.org> wrote:
>>>>>>
>>>>>> The following changes since commit 
>>>>>> 54e91d1523b412b4cff7cb36c898fa9dc133e886:
>>>>>>
>>>>>>    Merge tag 'pull-qapi-2025-02-10-v2' of https://repo.or.cz/qemu/ 
>>>>>> armbru into staging (2025-02-10 10:47:31 -0500)
>>>>>>
>>>>>> are available in the Git repository at:
>>>>>>
>>>>>>    https://github.com/philmd/qemu.git tags/hw-misc-20250210
>>>>>>
>>>>>> for you to fetch changes up to 1078a376932cc1d1c501ee3643fef329da6a189a:
>>>>>>
>>>>>>    hw/net/smc91c111: Ignore attempt to pop from empty RX fifo 
>>>>>> (2025-02-10 21:30:44 +0100)
>>>>>>
>>>>>> ----------------------------------------------------------------
>>>>>> Misc HW patches
>>>>>>
>>>>>> - Use qemu_hexdump_line() in TPM backend (Philippe)
>>>>>> - Make various Xilinx devices endianness configurable (Philippe)
>>>>>> - Remove magic number in APIC (Phil)
>>>>>> - Disable thread-level cache topology (Zhao)
>>>>>> - Xen QOM style cleanups (Bernhard)
>>>>>> - Introduce TYPE_DYNAMIC_SYS_BUS_DEVICE (Philippe)
>>>>>> - Invert logic of machine no_sdcard flag (Philippe)
>>>>>> - Housekeeping in MicroBlaze functional tests (Philippe)
>>>>>
>>>>> Please take a look at this CI failure:
>>>>> https://gitlab.com/qemu-project/qemu/-/jobs/9106591368
>>>>
>>>> Hmm I can not reproduce locally this error:
>>>>
>>>>    Exception: Asset cache is invalid and downloads disabled
>>>
>>> OK, I could reproduce by blowing my cache away.
>>>
>>> The problem seems in the "tests/functional: Have microblaze tests
>>> inherit common parent class" patch, which does:
>>>
>>> -class MicroblazeelMachine(QemuSystemTest):
>>> +class MicroblazeLittleEndianMachine(MicroblazeMachine):
>>>
>>> I presume, since MicroblazeLittleEndianMachine is no more a direct
>>> child of QemuSystemTest, then the ASSET_IMAGE_* aren't automatically
>>> downloaded.
>>
>> Well, apparently related to network failure:
>>
>> INFO:qemu-test:Downloading http://www.qemu-advent-calendar.org/2023/ 
>> download/day13.tar.gz to /Users/philmd/.cache/qemu/download/ 
>> b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22...
>> ERROR:qemu-test:Unable to download http://www.qemu-advent- 
>> calendar.org/2023/ download/day13.tar.gz: HTTP Error 403: Forbidden
>>
>> $ curl -v http://www.qemu-advent-calendar.org/2023/download/day13.tar.gz
>>  > GET /2023/download/day13.tar.gz HTTP/1.1
>> < HTTP/1.1 301 Moved Permanently
>> < Location: https://www.qemu-advent-calendar.org/2023/download/day13.tar.gz
>>
>> Using:
>>
>> -- >8 --
>> diff --git a/tests/functional/test_microblaze_s3adsp1800.py b/tests/ 
>> functional/test_microblaze_s3adsp1800.py
>> index 177c8a685bc..949e627c84a 100755
>> --- a/tests/functional/test_microblaze_s3adsp1800.py
>> +++ b/tests/functional/test_microblaze_s3adsp1800.py
>> @@ -24,3 +24,3 @@ class MicroblazeMachine(QemuSystemTest):
>>       ASSET_IMAGE_LE = Asset(
>> -        ('http://www.qemu-advent-calendar.org/2023/download/day13.tar.gz'),
>> + ('https://www.qemu-advent-calendar.org/2023/download/day13.tar.gz'),
>>
>> 'b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22')
>> ---
>>
>> I still get:
>>
>> INFO:qemu-test:Downloading https://www.qemu-advent-calendar.org/2023/ 
>> download/day13.tar.gz to /Users/philmd/.cache/qemu/download/ 
>> b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22...
>>
>> However:
>>
>> $ curl --http1.0 -v https://www.qemu-advent-calendar.org/2023/download/ 
>> day13.tar.gz
>>  > GET /2023/download/day13.tar.gz HTTP/1.0
>> < HTTP/1.0 200 OK
>> < Content-Length: 4752277
>> < Content-Type: application/gzip
>>
>> So I'm confused...
> 
> Looks like this also happens in test runs without your patches:
> 
>   https://gitlab.com/qemu-project/qemu/-/jobs/9108828196#L844
> 
> The test then silently gets skipped there (since we only fail hard for a 404 
> error now).
> 
> Maybe Eldon could comment why the downloads are blocked for python scripts 
> but not for curl downloads?
> 
> I the worst case, we have to mirror the asset to another place, I guess...

Ok, I've now also put the image here:

  https://qemu-advcal.gitlab.io/qac-best-of-multiarch/download/day05.tar.xz

(just had to use a different day number there since 13 was an image that I 
did not want to sacrifice in the best-of roaster).

So if the qemu-advent-calendar.org server keeps refusing the downloads, you 
should be able to use the mirrored tarball instead.

  HTH,
   Thomas
Eldon Stegall Feb. 12, 2025, 4:45 a.m. UTC | #8
On Wed, Feb 12, 2025 at 02:39:16AM +0100, Thomas Huth wrote:
> On 12/02/2025 01.41, Thomas Huth wrote:
> > On 11/02/2025 20.03, Philippe Mathieu-Daudé wrote:
> >> On 11/2/25 19:53, Philippe Mathieu-Daudé wrote:
> >>> On 11/2/25 19:48, Philippe Mathieu-Daudé wrote:
> >>>> On 11/2/25 19:26, Stefan Hajnoczi wrote:
> >>>>> On Mon, Feb 10, 2025 at 3:43 PM Philippe Mathieu-Daudé
> >>>>> <philmd@linaro.org> wrote:
> >>>>>>
> >>>>>> The following changes since commit 
> >>>>>> 54e91d1523b412b4cff7cb36c898fa9dc133e886:
> >>>>>>
> >>>>>>    Merge tag 'pull-qapi-2025-02-10-v2' of https://repo.or.cz/qemu/ 
> >>>>>> armbru into staging (2025-02-10 10:47:31 -0500)
> >>>>>>
> >>>>>> are available in the Git repository at:
> >>>>>>
> >>>>>>    https://github.com/philmd/qemu.git tags/hw-misc-20250210
> >>>>>>
> >>>>>> for you to fetch changes up to 1078a376932cc1d1c501ee3643fef329da6a189a:
> >>>>>>
> >>>>>>    hw/net/smc91c111: Ignore attempt to pop from empty RX fifo 
> >>>>>> (2025-02-10 21:30:44 +0100)
> >>>>>>
> >>>>>> ----------------------------------------------------------------
> >>>>>> Misc HW patches
> >>>>>>
> >>>>>> - Use qemu_hexdump_line() in TPM backend (Philippe)
> >>>>>> - Make various Xilinx devices endianness configurable (Philippe)
> >>>>>> - Remove magic number in APIC (Phil)
> >>>>>> - Disable thread-level cache topology (Zhao)
> >>>>>> - Xen QOM style cleanups (Bernhard)
> >>>>>> - Introduce TYPE_DYNAMIC_SYS_BUS_DEVICE (Philippe)
> >>>>>> - Invert logic of machine no_sdcard flag (Philippe)
> >>>>>> - Housekeeping in MicroBlaze functional tests (Philippe)
> >>>>>
> >>>>> Please take a look at this CI failure:
> >>>>> https://gitlab.com/qemu-project/qemu/-/jobs/9106591368
> >>>>
> >>>> Hmm I can not reproduce locally this error:
> >>>>
> >>>>    Exception: Asset cache is invalid and downloads disabled
> >>>
> >>> OK, I could reproduce by blowing my cache away.
> >>>
> >>> The problem seems in the "tests/functional: Have microblaze tests
> >>> inherit common parent class" patch, which does:
> >>>
> >>> -class MicroblazeelMachine(QemuSystemTest):
> >>> +class MicroblazeLittleEndianMachine(MicroblazeMachine):
> >>>
> >>> I presume, since MicroblazeLittleEndianMachine is no more a direct
> >>> child of QemuSystemTest, then the ASSET_IMAGE_* aren't automatically
> >>> downloaded.
> >>
> >> Well, apparently related to network failure:
> >>
> >> INFO:qemu-test:Downloading http://www.qemu-advent-calendar.org/2023/ 
> >> download/day13.tar.gz to /Users/philmd/.cache/qemu/download/ 
> >> b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22...
> >> ERROR:qemu-test:Unable to download http://www.qemu-advent- 
> >> calendar.org/2023/ download/day13.tar.gz: HTTP Error 403: Forbidden
> >>
> >> $ curl -v http://www.qemu-advent-calendar.org/2023/download/day13.tar.gz
> >>  > GET /2023/download/day13.tar.gz HTTP/1.1
> >> < HTTP/1.1 301 Moved Permanently
> >> < Location: https://www.qemu-advent-calendar.org/2023/download/day13.tar.gz
> >>
> >> Using:
> >>
> >> -- >8 --
> >> diff --git a/tests/functional/test_microblaze_s3adsp1800.py b/tests/ 
> >> functional/test_microblaze_s3adsp1800.py
> >> index 177c8a685bc..949e627c84a 100755
> >> --- a/tests/functional/test_microblaze_s3adsp1800.py
> >> +++ b/tests/functional/test_microblaze_s3adsp1800.py
> >> @@ -24,3 +24,3 @@ class MicroblazeMachine(QemuSystemTest):
> >>       ASSET_IMAGE_LE = Asset(
> >> -        ('http://www.qemu-advent-calendar.org/2023/download/day13.tar.gz'),
> >> + ('https://www.qemu-advent-calendar.org/2023/download/day13.tar.gz'),
> >>
> >> 'b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22')
> >> ---
> >>
> >> I still get:
> >>
> >> INFO:qemu-test:Downloading https://www.qemu-advent-calendar.org/2023/ 
> >> download/day13.tar.gz to /Users/philmd/.cache/qemu/download/ 
> >> b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22...
> >>
> >> However:
> >>
> >> $ curl --http1.0 -v https://www.qemu-advent-calendar.org/2023/download/ 
> >> day13.tar.gz
> >>  > GET /2023/download/day13.tar.gz HTTP/1.0
> >> < HTTP/1.0 200 OK
> >> < Content-Length: 4752277
> >> < Content-Type: application/gzip
> >>
> >> So I'm confused...
> > 
> > Looks like this also happens in test runs without your patches:
> > 
> >   https://gitlab.com/qemu-project/qemu/-/jobs/9108828196#L844
> > 
> > The test then silently gets skipped there (since we only fail hard for a 404 
> > error now).
> > 
> > Maybe Eldon could comment why the downloads are blocked for python scripts 
> > but not for curl downloads?
> > 
> > I the worst case, we have to mirror the asset to another place, I guess...
> 
> Ok, I've now also put the image here:
> 
>   https://qemu-advcal.gitlab.io/qac-best-of-multiarch/download/day05.tar.xz
> 
> (just had to use a different day number there since 13 was an image that I 
> did not want to sacrifice in the best-of roaster).
> 
> So if the qemu-advent-calendar.org server keeps refusing the downloads, you 
> should be able to use the mirrored tarball instead.
> 
>   HTH,
>    Thomas
> 
>

Hello! I recently switched some of the underlying object storage for
sites including the advent calendar, and this was a (very) unintended
consequence. I have now switched to a good old apache upstream for
these, so I would not expect this error to recur. Also, looking to make
another attempt at a calendar for 2025 (end of 2024 was a complicated
time for me). Let me know if you'd like to produce/contribute!

Thanks,
Eldon
Daniel P. Berrangé Feb. 12, 2025, 9:10 a.m. UTC | #9
On Tue, Feb 11, 2025 at 07:53:32PM +0100, Philippe Mathieu-Daudé wrote:
> On 11/2/25 19:48, Philippe Mathieu-Daudé wrote:
> > On 11/2/25 19:26, Stefan Hajnoczi wrote:
> > > On Mon, Feb 10, 2025 at 3:43 PM Philippe Mathieu-Daudé
> > > <philmd@linaro.org> wrote:
> > > > 
> > > > The following changes since commit
> > > > 54e91d1523b412b4cff7cb36c898fa9dc133e886:
> > > > 
> > > >    Merge tag 'pull-qapi-2025-02-10-v2' of
> > > > https://repo.or.cz/qemu/ armbru into staging (2025-02-10
> > > > 10:47:31 -0500)
> > > > 
> > > > are available in the Git repository at:
> > > > 
> > > >    https://github.com/philmd/qemu.git tags/hw-misc-20250210
> > > > 
> > > > for you to fetch changes up to 1078a376932cc1d1c501ee3643fef329da6a189a:
> > > > 
> > > >    hw/net/smc91c111: Ignore attempt to pop from empty RX fifo
> > > > (2025-02-10 21:30:44 +0100)
> > > > 
> > > > ----------------------------------------------------------------
> > > > Misc HW patches
> > > > 
> > > > - Use qemu_hexdump_line() in TPM backend (Philippe)
> > > > - Make various Xilinx devices endianness configurable (Philippe)
> > > > - Remove magic number in APIC (Phil)
> > > > - Disable thread-level cache topology (Zhao)
> > > > - Xen QOM style cleanups (Bernhard)
> > > > - Introduce TYPE_DYNAMIC_SYS_BUS_DEVICE (Philippe)
> > > > - Invert logic of machine no_sdcard flag (Philippe)
> > > > - Housekeeping in MicroBlaze functional tests (Philippe)
> > > 
> > > Please take a look at this CI failure:
> > > https://gitlab.com/qemu-project/qemu/-/jobs/9106591368
> > 
> > Hmm I can not reproduce locally this error:
> > 
> >    Exception: Asset cache is invalid and downloads disabled
> 
> OK, I could reproduce by blowing my cache away.
> 
> The problem seems in the "tests/functional: Have microblaze tests
> inherit common parent class" patch, which does:
> 
> -class MicroblazeelMachine(QemuSystemTest):
> +class MicroblazeLittleEndianMachine(MicroblazeMachine):
> 
> I presume, since MicroblazeLittleEndianMachine is no more a direct
> child of QemuSystemTest, then the ASSET_IMAGE_* aren't automatically
> downloaded.

Yes, my code assumes all assets are on the leaf test classes. I'll
look at a fix since it is easy enough to check parent classes too.

With regards,
Daniel
Philippe Mathieu-Daudé Feb. 12, 2025, 11:57 a.m. UTC | #10
On 12/2/25 10:10, Daniel P. Berrangé wrote:
> On Tue, Feb 11, 2025 at 07:53:32PM +0100, Philippe Mathieu-Daudé wrote:
>> On 11/2/25 19:48, Philippe Mathieu-Daudé wrote:
>>> On 11/2/25 19:26, Stefan Hajnoczi wrote:
>>>> On Mon, Feb 10, 2025 at 3:43 PM Philippe Mathieu-Daudé
>>>> <philmd@linaro.org> wrote:
>>>>>
>>>>> The following changes since commit
>>>>> 54e91d1523b412b4cff7cb36c898fa9dc133e886:
>>>>>
>>>>>     Merge tag 'pull-qapi-2025-02-10-v2' of
>>>>> https://repo.or.cz/qemu/ armbru into staging (2025-02-10
>>>>> 10:47:31 -0500)
>>>>>
>>>>> are available in the Git repository at:
>>>>>
>>>>>     https://github.com/philmd/qemu.git tags/hw-misc-20250210
>>>>>
>>>>> for you to fetch changes up to 1078a376932cc1d1c501ee3643fef329da6a189a:
>>>>>
>>>>>     hw/net/smc91c111: Ignore attempt to pop from empty RX fifo
>>>>> (2025-02-10 21:30:44 +0100)
>>>>>
>>>>> ----------------------------------------------------------------
>>>>> Misc HW patches
>>>>>
>>>>> - Use qemu_hexdump_line() in TPM backend (Philippe)
>>>>> - Make various Xilinx devices endianness configurable (Philippe)
>>>>> - Remove magic number in APIC (Phil)
>>>>> - Disable thread-level cache topology (Zhao)
>>>>> - Xen QOM style cleanups (Bernhard)
>>>>> - Introduce TYPE_DYNAMIC_SYS_BUS_DEVICE (Philippe)
>>>>> - Invert logic of machine no_sdcard flag (Philippe)
>>>>> - Housekeeping in MicroBlaze functional tests (Philippe)
>>>>
>>>> Please take a look at this CI failure:
>>>> https://gitlab.com/qemu-project/qemu/-/jobs/9106591368
>>>
>>> Hmm I can not reproduce locally this error:
>>>
>>>     Exception: Asset cache is invalid and downloads disabled
>>
>> OK, I could reproduce by blowing my cache away.
>>
>> The problem seems in the "tests/functional: Have microblaze tests
>> inherit common parent class" patch, which does:
>>
>> -class MicroblazeelMachine(QemuSystemTest):
>> +class MicroblazeLittleEndianMachine(MicroblazeMachine):
>>
>> I presume, since MicroblazeLittleEndianMachine is no more a direct
>> child of QemuSystemTest, then the ASSET_IMAGE_* aren't automatically
>> downloaded.
> 
> Yes, my code assumes all assets are on the leaf test classes. I'll
> look at a fix since it is easy enough to check parent classes too.

Actually your code works fine :) I mis-interpreted the network issue:

DEBUG:qemu-test:Extract 
/Users/philmd/.cache/qemu/download/b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22 
format=Nonesub_dir=None member=None
INFO:qemu-test:Downloading 
http://www.qemu-advent-calendar.org/2023/download/day13.tar.gz to 
/Users/philmd/.cache/qemu/download/b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22...
DEBUG:qemu-test:Unable to set xattr on 
/Users/philmd/.cache/qemu/download/b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22.download: 
module 'os' has no attribute 'setxattr'
INFO:qemu-test:Cached 
http://www.qemu-advent-calendar.org/2023/download/day13.tar.gz at 
/Users/philmd/.cache/qemu/download/b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22
Thomas Huth Feb. 12, 2025, 7:31 p.m. UTC | #11
On 12/02/2025 05.45, Eldon Stegall wrote:
> On Wed, Feb 12, 2025 at 02:39:16AM +0100, Thomas Huth wrote:
>> On 12/02/2025 01.41, Thomas Huth wrote:
>>> On 11/02/2025 20.03, Philippe Mathieu-Daudé wrote:
>>>> On 11/2/25 19:53, Philippe Mathieu-Daudé wrote:
>>>>> On 11/2/25 19:48, Philippe Mathieu-Daudé wrote:
[...]
>>>> Well, apparently related to network failure:
>>>>
>>>> INFO:qemu-test:Downloading http://www.qemu-advent-calendar.org/2023/
>>>> download/day13.tar.gz to /Users/philmd/.cache/qemu/download/
>>>> b9b3d43c5dd79db88ada495cc6e0d1f591153fe41355e925d791fbf44de50c22...
>>>> ERROR:qemu-test:Unable to download http://www.qemu-advent-
>>>> calendar.org/2023/ download/day13.tar.gz: HTTP Error 403: Forbidden
[...]
>>>> However:
>>>>
>>>> $ curl --http1.0 -v https://www.qemu-advent-calendar.org/2023/download/
>>>> day13.tar.gz
>>>>   > GET /2023/download/day13.tar.gz HTTP/1.0
>>>> < HTTP/1.0 200 OK
>>>> < Content-Length: 4752277
>>>> < Content-Type: application/gzip
>>>>
>>>> So I'm confused...
>>>
>>> Looks like this also happens in test runs without your patches:
>>>
>>>    https://gitlab.com/qemu-project/qemu/-/jobs/9108828196#L844
>>>
>>> The test then silently gets skipped there (since we only fail hard for a 404
>>> error now).
>>>
>>> Maybe Eldon could comment why the downloads are blocked for python scripts
>>> but not for curl downloads?
>>>
>>> I the worst case, we have to mirror the asset to another place, I guess...
[...]
> Hello! I recently switched some of the underlying object storage for
> sites including the advent calendar, and this was a (very) unintended
> consequence. I have now switched to a good old apache upstream for
> these, so I would not expect this error to recur.

Thanks a lot for the quick fix!

> Also, looking to make
> another attempt at a calendar for 2025 (end of 2024 was a complicated
> time for me). Let me know if you'd like to produce/contribute!

Sure, count me in for an image or two!

  Thomas