mbox series

[for-3.2,v7,0/6] Connect a PCIe host and graphics support to RISC-V

Message ID cover.1542819633.git.alistair.francis@wdc.com (mailing list archive)
Headers show
Series Connect a PCIe host and graphics support to RISC-V | expand

Message

Alistair Francis Nov. 21, 2018, 5:02 p.m. UTC
V7:
 - Fix the GPEX memory mapping thanks to Bin Meng
 - Fix the interrupt mapping thanks to Logan Gunthorpe
V6:
 - Fix the interrupt issue for the GPEX device
V5:
 - Rebase
 - Include pci.mak in the default configs
V4:
 - Fix the spike device tree
 - Don't use stdvga device
V3:
 - Remove Makefile config changes
 - Connect a network adapter to the virt device
V2:
 - Use the gpex PCIe host for virt
 - Add support for SiFive U PCIe


Alistair Francis (6):
  hw/riscv/virt: Increase the number of interrupts
  hw/riscv/virt: Adjust memory layout spacing
  hw/riscv/virt: Connect the gpex PCIe
  riscv: Enable VGA and PCIE_VGA
  hw/riscv/sifive_u: Connect the Xilinx PCIe
  hw/riscv/virt: Connect a VirtIO net PCIe device

 default-configs/riscv32-softmmu.mak |  10 +-
 default-configs/riscv64-softmmu.mak |  10 +-
 hw/riscv/sifive_u.c                 |  64 +++++++++++
 hw/riscv/virt.c                     | 161 ++++++++++++++++++++++++++--
 include/hw/riscv/sifive_u.h         |   4 +-
 include/hw/riscv/virt.h             |  15 ++-
 6 files changed, 250 insertions(+), 14 deletions(-)

Comments

Andrea Bolognani Nov. 22, 2018, 10:59 a.m. UTC | #1
On Wed, 2018-11-21 at 17:02 +0000, Alistair Francis wrote:
> V7:
>  - Fix the GPEX memory mapping thanks to Bin Meng
>  - Fix the interrupt mapping thanks to Logan Gunthorpe
> 
[...]
> 
> Alistair Francis (6):
>   hw/riscv/virt: Increase the number of interrupts
>   hw/riscv/virt: Adjust memory layout spacing
>   hw/riscv/virt: Connect the gpex PCIe
>   riscv: Enable VGA and PCIE_VGA
>   hw/riscv/sifive_u: Connect the Xilinx PCIe
>   hw/riscv/virt: Connect a VirtIO net PCIe device

Using QEMU master + these patches, libvirt master + my own patches,
a disk image from [1] and the bbl from [2], I was able to run a pure
PCI RISC-V guest, including connecting to it through ssh and
performing a system update using dnf; based on these results, the
series gets a big old

  Tested-by: Andrea Bolognani <abologna@redhat.com>


[1] https://fedora-riscv.tranquillity.se/koji/
    Unfortunately the site is broken at the moment :(
[2] https://fedorapeople.org/groups/risc-v/disk-images/
    PCI support is included in the bbl now! \o/
Alistair Francis Nov. 26, 2018, 4:03 p.m. UTC | #2
On Thu, Nov 22, 2018 at 2:59 AM Andrea Bolognani <abologna@redhat.com> wrote:
>
> On Wed, 2018-11-21 at 17:02 +0000, Alistair Francis wrote:
> > V7:
> >  - Fix the GPEX memory mapping thanks to Bin Meng
> >  - Fix the interrupt mapping thanks to Logan Gunthorpe
> >
> [...]
> >
> > Alistair Francis (6):
> >   hw/riscv/virt: Increase the number of interrupts
> >   hw/riscv/virt: Adjust memory layout spacing
> >   hw/riscv/virt: Connect the gpex PCIe
> >   riscv: Enable VGA and PCIE_VGA
> >   hw/riscv/sifive_u: Connect the Xilinx PCIe
> >   hw/riscv/virt: Connect a VirtIO net PCIe device
>
> Using QEMU master + these patches, libvirt master + my own patches,
> a disk image from [1] and the bbl from [2], I was able to run a pure
> PCI RISC-V guest, including connecting to it through ssh and
> performing a system update using dnf; based on these results, the
> series gets a big old
>
>   Tested-by: Andrea Bolognani <abologna@redhat.com>

Great! Thanks for testing.

Alistair

>
>
> [1] https://fedora-riscv.tranquillity.se/koji/
>     Unfortunately the site is broken at the moment :(
> [2] https://fedorapeople.org/groups/risc-v/disk-images/
>     PCI support is included in the bbl now! \o/
> --
> Andrea Bolognani / Red Hat / Virtualization
>
Palmer Dabbelt Nov. 26, 2018, 7:34 p.m. UTC | #3
On Thu, 22 Nov 2018 02:59:18 PST (-0800), abologna@redhat.com wrote:
> On Wed, 2018-11-21 at 17:02 +0000, Alistair Francis wrote:
>> V7:
>>  - Fix the GPEX memory mapping thanks to Bin Meng
>>  - Fix the interrupt mapping thanks to Logan Gunthorpe
>>
> [...]
>>
>> Alistair Francis (6):
>>   hw/riscv/virt: Increase the number of interrupts
>>   hw/riscv/virt: Adjust memory layout spacing
>>   hw/riscv/virt: Connect the gpex PCIe
>>   riscv: Enable VGA and PCIE_VGA
>>   hw/riscv/sifive_u: Connect the Xilinx PCIe
>>   hw/riscv/virt: Connect a VirtIO net PCIe device
>
> Using QEMU master + these patches, libvirt master + my own patches,
> a disk image from [1] and the bbl from [2], I was able to run a pure
> PCI RISC-V guest, including connecting to it through ssh and
> performing a system update using dnf; based on these results, the
> series gets a big old
>
>   Tested-by: Andrea Bolognani <abologna@redhat.com>
>
>
> [1] https://fedora-riscv.tranquillity.se/koji/
>     Unfortunately the site is broken at the moment :(

FWIW, I get my images from here

    https://fedorapeople.org/groups/risc-v/disk-images/

which is linked to from the Fedora wiki

    https://fedoraproject.org/wiki/Architectures/RISC-V/Disk_images

the stage4 there is quite old (April, 2018), is yours newer?  I'm having some 
problems updating my disk image and since I'm not a Fedora user I have no idea 
what's going on.

> [2] https://fedorapeople.org/groups/risc-v/disk-images/
>     PCI support is included in the bbl now! \o/

Great!  I've yet to figure out how to get all the PCI stuff working, but I was 
trying to go with a virgl-based GPU which I suspect is broken for other 
reasons.  What sort of devices did you attach?
Guenter Roeck Nov. 26, 2018, 9:33 p.m. UTC | #4
On Mon, Nov 26, 2018 at 11:34:58AM -0800, Palmer Dabbelt wrote:
> On Thu, 22 Nov 2018 02:59:18 PST (-0800), abologna@redhat.com wrote:
> > On Wed, 2018-11-21 at 17:02 +0000, Alistair Francis wrote:
> >> V7:
> >>  - Fix the GPEX memory mapping thanks to Bin Meng
> >>  - Fix the interrupt mapping thanks to Logan Gunthorpe
> >>
> > [...]
> >>
> >> Alistair Francis (6):
> >>   hw/riscv/virt: Increase the number of interrupts
> >>   hw/riscv/virt: Adjust memory layout spacing
> >>   hw/riscv/virt: Connect the gpex PCIe
> >>   riscv: Enable VGA and PCIE_VGA
> >>   hw/riscv/sifive_u: Connect the Xilinx PCIe
> >>   hw/riscv/virt: Connect a VirtIO net PCIe device
> >
> > Using QEMU master + these patches, libvirt master + my own patches,
> > a disk image from [1] and the bbl from [2], I was able to run a pure
> > PCI RISC-V guest, including connecting to it through ssh and
> > performing a system update using dnf; based on these results, the
> > series gets a big old
> >
> >   Tested-by: Andrea Bolognani <abologna@redhat.com>
> >
> >
> > [1] https://fedora-riscv.tranquillity.se/koji/
> >     Unfortunately the site is broken at the moment :(
> 
> FWIW, I get my images from here
> 
>     https://fedorapeople.org/groups/risc-v/disk-images/
> 
> which is linked to from the Fedora wiki
> 
>     https://fedoraproject.org/wiki/Architectures/RISC-V/Disk_images
> 
> the stage4 there is quite old (April, 2018), is yours newer?  I'm having some 
> problems updating my disk image and since I'm not a Fedora user I have no idea 
> what's going on.
> 
> > [2] https://fedorapeople.org/groups/risc-v/disk-images/
> >     PCI support is included in the bbl now! \o/
> 
> Great!  I've yet to figure out how to get all the PCI stuff working, but I was 
> trying to go with a virgl-based GPU which I suspect is broken for other 
> reasons.  What sort of devices did you attach?

FWIW, here is my list of tests.

Building riscv:virt:defconfig:initrd ... running ....... passed
Building riscv:virt:defconfig:virtio-blk:rootfs ... running ........ passed
Building riscv:virt:defconfig:virtio:rootfs ... running ........ passed
Building riscv:virt:defconfig:virtio-pci:rootfs ... running ........ passed
Building riscv:virt:defconfig:mmc:rootfs ... running ........ passed
Building riscv:virt:defconfig:nvme:rootfs ... running ........ passed
Building riscv:virt:defconfig:usb-ohci:rootfs ... running ........ passed
Building riscv:virt:defconfig:usb-ehci:rootfs ... running ........ passed
Building riscv:virt:defconfig:usb-xhci:rootfs ... running ......... passed
Building riscv:virt:defconfig:usb-uas-ehci:rootfs ... running ......... passed
Building riscv:virt:defconfig:usb-uas-xhci:rootfs ... running ......... passed
Building riscv:virt:defconfig:scsi[53C810]:rootfs ... running ........... passed
Building riscv:virt:defconfig:scsi[53C895A]:rootfs ... running ......... passed
Building riscv:virt:defconfig:scsi[AM53C974]:rootfs ... skipped
Building riscv:virt:defconfig:scsi[DC395]:rootfs ... skipped
Building riscv:virt:defconfig:scsi[MEGASAS]:rootfs ... running ......... passed
Building riscv:virt:defconfig:scsi[MEGASAS2]:rootfs ... running ........ passed
Building riscv:virt:defconfig:scsi[FUSION]:rootfs ... running ....... passed
Building riscv:virt:defconfig:scsi[virtio]:rootfs ... running ....... passed
Building riscv:virt:defconfig:scsi[virtio-pci]:rootfs ... running ........ passed

The skipped tests work as well, but the emulation is flaky and fails once
in a while. This is obviously with usb enabled.

Guenter
Andrea Bolognani Nov. 27, 2018, 12:40 p.m. UTC | #5
On Mon, 2018-11-26 at 11:34 -0800, Palmer Dabbelt wrote:
> FWIW, I get my images from here
> 
>     https://fedorapeople.org/groups/risc-v/disk-images/
> 
> which is linked to from the Fedora wiki
> 
>     https://fedoraproject.org/wiki/Architectures/RISC-V/Disk_images
> 
> the stage4 there is quite old (April, 2018), is yours newer?  I'm having some 
> problems updating my disk image and since I'm not a Fedora user I have no idea 
> what's going on.

Yeah, we have nightly built images now - they come in several
flavors, even!

  http://185.97.32.145/koji/tasks?state=closed&view=flat&method=createAppliance&order=-id

Note that there are some issues with the server configuration and
the HSTS preload list shipped with browsers not playing nice with
each other, so you'll have to tweak URLs a bit before you can
actually download images, basically

  s,https://fedora-riscv.tranquillity.se,http://185.97.32.145,

I wonder if it even makes sense to keep the old stage4 images
around... And someone should definitely sweep through the wiki
and make sure we don't point to obsolete information and all URLs
are up to date. Maybe I should create that Fedora Account after
all O:-)

> > [2] https://fedorapeople.org/groups/risc-v/disk-images/
> >     PCI support is included in the bbl now! \o/
> 
> Great!  I've yet to figure out how to get all the PCI stuff working, but I was 
> trying to go with a virgl-based GPU which I suspect is broken for other 
> reasons.  What sort of devices did you attach?

I just tried today, and virtio-gpu doesn't work for me either:
during boot I get

  [drm] radeon kernel modesetting enabled.
  [drm] pci: virtio-gpu-pci detected at 0000:00:05.0
  [drm] virgl 3d acceleration not supported by host
  [TTM] Zone  kernel: Available graphics memory: 993416 kiB
  [TTM] Initializing pool allocator
  [TTM] Initializing DMA pool allocator
  [drm] number of scanouts: 1
  [drm] number of cap sets: 0
  ------------[ cut here ]------------
  WARNING: CPU: 3 PID: 1 at lib/scatterlist.c:275 __sg_alloc_table+0xba/0xbe
  Modules linked in:
  CPU: 3 PID: 1 Comm: swapper/0 Not tainted 4.19.0-rc8 #1
  Call Trace:
  [<ffffffe0000383ec>] walk_stackframe+0x0/0xa0
  [<ffffffe000038588>] show_stack+0x2a/0x34
  [<ffffffe0006b12b2>] dump_stack+0x62/0x7c
  [<ffffffe00003c498>] __warn+0xd2/0xe4
  [<ffffffe00003c544>] warn_slowpath_null+0x30/0x3e
  [<ffffffe0002a1646>] __sg_alloc_table+0xba/0xbe
  [<ffffffe0002a18c8>] sg_alloc_table+0x28/0xa6
  [<ffffffe0002a1a1a>] __sg_alloc_table_from_pages+0xd4/0x1da
  [<ffffffe0002a1b30>] sg_alloc_table_from_pages+0x10/0x18
  [<ffffffe00042c87c>] virtio_gpu_object_get_sg_table+0x90/0xa6
  [<ffffffe00042bdfc>] virtio_gpu_object_attach+0x12c/0x14c
  [<ffffffe00042a1aa>] virtio_gpufb_create+0xe4/0x1c0
  [<ffffffe0003345f2>] __drm_fb_helper_initial_config_and_unlock+0x198/0x334
  [<ffffffe000334c5a>] drm_fb_helper_initial_config+0x2e/0x3a
  [<ffffffe00042a3f6>] virtio_gpu_fbdev_init+0xa6/0xce
  [<ffffffe000429556>] virtio_gpu_driver_load+0x36e/0x3be
  [<ffffffe00033a932>] drm_dev_register+0xfe/0x162
  [<ffffffe000429968>] drm_virtio_init+0x48/0x158
  [<ffffffe000429164>] virtio_gpu_probe+0x32/0x40
  [<ffffffe0002f7e58>] virtio_dev_probe+0x10e/0x196
  [<ffffffe00043546c>] really_probe+0x17e/0x1ea
  [<ffffffe000435654>] driver_probe_device+0x7a/0x90
  [<ffffffe000435702>] __driver_attach+0x98/0x9a
  [<ffffffe000433bc2>] bus_for_each_dev+0x4a/0x72
  [<ffffffe000434f28>] driver_attach+0x1a/0x22
  [<ffffffe000434b06>] bus_add_driver+0x156/0x1b8
  [<ffffffe000435c88>] driver_register+0x3a/0xd0
  [<ffffffe0002f7f00>] register_virtio_driver+0x20/0x2e
  [<ffffffe000012d92>] virtio_gpu_driver_init+0x18/0x20
  [<ffffffe00003664a>] do_one_initcall+0x2c/0x10a
  [<ffffffe000000a90>] kernel_init_freeable+0x11e/0x1ae
  [<ffffffe0006c2304>] kernel_init+0x12/0xf0
  [<ffffffe000037472>] ret_from_exception+0x0/0xc
  ---[ end trace 161c50bff518fcd6 ]---
  [drm] Initialized virtio_gpu 0.0.1 0 for virtio5 on minor 0

and the graphical console is not initialized.

The libvirt guest XML I'm using is

  <domain type='qemu'>
    <name>riscv</name>
    <memory unit='GiB'>2</memory>
    <vcpu>4</vcpu>
    <os>
      <type arch='riscv64' machine='virt'>hvm</type>
      <kernel>/var/lib/libvirt/images/riscv.bbl</kernel>
      <cmdline>root=/dev/vda1</cmdline>
    </os>
    <devices>
      <emulator>/usr/bin/qemu-system-riscv64</emulator>
      <disk type='file' device='disk'>
        <driver name='qemu' type='raw'/>
        <source file='/var/lib/libvirt/images/riscv.raw'/>
        <target dev='vda' bus='virtio'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
      </disk>
      <controller type='pci' model='pcie-root'/>
      <controller type='virtio-serial'>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </controller>
      <interface type='network'>
        <source network='default'/>
        <model type='virtio'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
      </interface>
      <serial type='pty'/>
      <channel type='unix'>
        <target type='virtio' name='org.qemu.guest_agent.0'/>
      </channel>
      <graphics type='spice'/>
      <video>
        <model type='virtio'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
      </video>
      <memballoon model='virtio'>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
      </memballoon>
      <rng model='virtio'>
        <backend model='random'>/dev/urandom</backend>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
      </rng>
    </devices>
  </domain>

but you're going to need a libvirt built from

  https://github.com/andreabolognani/libvirt/tree/riscv-pci

in order to actually use it.

I think the BBL I'm using doesn't have ballooning support, and I
haven't bothered trying it either, but as reported earlier everything
else (disk, networking, RNG, shutdown via qemu-guest-agent) works
just fine.