| Message ID | 20230307171750.2293175-1-dwmw2@infradead.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | Enable PV backends with Xen/KVM emulation | expand |
On 07/03/2023 17:17, David Woodhouse wrote: > Following on from the basic platform support which has already been > merged, here's phase 2 which wires up the XenBus and PV back ends. > > It starts with a basic single-tenant internal implementation of a > XenStore, with a copy-on-write tree, watches, transactions, quotas. > > Then we introduce operations tables for the grant table, event channel, > foreignmen and xenstore operations so that in addition to using the Xen > libraries for those, QEMU can use its internal emulated versions. > > A little bit of cleaning up of header files, and we can enable the build > of xen-bus in the CONFIG_XEN_EMU build, and run a Xen guest with an > actual PV disk... > > qemu-system-x86_64 -serial mon:stdio -M q35 -display none -m 1G -smp 2 \ > -accel kvm,xen-version=0x4000e,kernel-irqchip=split \ > -kernel bzImage -append "console=ttyS0 root=/dev/xvda1 selinux=0" \ > -drive file=/var/lib/libvirt/images/fedora28.qcow2,if=none,id=disk \ > -device xen-disk,drive=disk,vdev=xvda > > The main thing that isn't working here is migration. I've implemented it > for the internal xenstore and the unit tests exercise it, but the > existing PV back ends don't support it, perhaps partly because support > for guest transparent live migration support isn't upstream in Xen yet. > So the disk doesn't come back correctly after migration. I'm content > with that for 8.0 though, and we just mark the emulated XenStore device > as unmigratable to prevent users from trying. > > The other pre-existing constraint is that only the block back end has > yet been ported to the "new" XenBus infrastructure, and is actually > capable of creating its own backend nodes. Again, I can live with > that for 8.0. Maybe this will motivate us to finally get round to > converting the rest off XenLegacyBackend and killing it. > > We also don't have a simple way to perform grant mapping of multiple > guest pages to contiguous addresses, as we can under real Xen. So we > don't advertise max-ring-page-order for xen-disk in the emulated mode. > Fixing that — if we actually want to — would probably require mapping > RAM from an actual backing store object, so that it can be mapped again > at a different location for the PV back end to see. > > v2: https://git.infradead.org/users/dwmw2/qemu.git/shortlog/refs/heads/xenfv-2 > > • Full set of reviewed-by tags from Paul (and associated minor fixes). > > • Disable migration for emulated XenStore device. > > • Update docs and add MAINTAINERS entry. > > v1: https://lore.kernel.org/qemu-devel/20230302153435.1170111-1-dwmw2@infradead.org/ > https://git.infradead.org/users/dwmw2/qemu.git/shortlog/refs/heads/xenfv-1 > > David Woodhouse (23): > hw/xen: Add xenstore wire implementation and implementation stubs > hw/xen: Add basic XenStore tree walk and write/read/directory support > hw/xen: Implement XenStore watches > hw/xen: Implement XenStore transactions > hw/xen: Watches on XenStore transactions > hw/xen: Implement core serialize/deserialize methods for xenstore_impl > hw/xen: Add evtchn operations to allow redirection to internal emulation > hw/xen: Add gnttab operations to allow redirection to internal emulation > hw/xen: Pass grant ref to gnttab unmap operation > hw/xen: Add foreignmem operations to allow redirection to internal emulation > hw/xen: Move xenstore_store_pv_console_info to xen_console.c > hw/xen: Use XEN_PAGE_SIZE in PV backend drivers > hw/xen: Rename xen_common.h to xen_native.h > hw/xen: Build PV backend drivers for CONFIG_XEN_BUS > hw/xen: Only advertise ring-page-order for xen-block if gnttab supports it > hw/xen: Hook up emulated implementation for event channel operations > hw/xen: Add emulated implementation of grant table operations > hw/xen: Add emulated implementation of XenStore operations > hw/xen: Map guest XENSTORE_PFN grant in emulated Xenstore > hw/xen: Implement soft reset for emulated gnttab > i386/xen: Initialize Xen backends from pc_basic_device_init() for emulation > MAINTAINERS: Add entry for Xen on KVM emulation > docs: Update Xen-on-KVM documentation for PV disk support > > Paul Durrant (4): > hw/xen: Implement XenStore permissions > hw/xen: Create initial XenStore nodes > hw/xen: Add xenstore operations to allow redirection to internal emulation > hw/xen: Avoid crash when backend watch fires too early > > MAINTAINERS | 9 + > accel/xen/xen-all.c | 69 +- > docs/system/i386/xen.rst | 30 +- > hw/9pfs/meson.build | 2 +- > hw/9pfs/xen-9p-backend.c | 32 +- > hw/block/dataplane/meson.build | 2 +- > hw/block/dataplane/xen-block.c | 12 +- > hw/block/meson.build | 2 +- > hw/block/xen-block.c | 12 +- > hw/char/meson.build | 2 +- > hw/char/xen_console.c | 57 +- > hw/display/meson.build | 2 +- > hw/display/xenfb.c | 32 +- > hw/i386/kvm/meson.build | 1 + > hw/i386/kvm/trace-events | 15 + > hw/i386/kvm/xen_evtchn.c | 15 + > hw/i386/kvm/xen_gnttab.c | 325 ++++- > hw/i386/kvm/xen_gnttab.h | 1 + > hw/i386/kvm/xen_xenstore.c | 1251 +++++++++++++++- > hw/i386/kvm/xenstore_impl.c | 1927 +++++++++++++++++++++++++ > hw/i386/kvm/xenstore_impl.h | 63 + > hw/i386/pc.c | 7 + > hw/i386/pc_piix.c | 4 +- > hw/i386/xen/xen-hvm.c | 38 +- > hw/i386/xen/xen-mapcache.c | 2 +- > hw/i386/xen/xen_platform.c | 7 +- > hw/net/xen_nic.c | 25 +- > hw/usb/meson.build | 2 +- > hw/usb/xen-usb.c | 29 +- > hw/xen/meson.build | 6 +- > hw/xen/trace-events | 2 +- > hw/xen/xen-bus-helper.c | 62 +- > hw/xen/xen-bus.c | 411 +----- > hw/xen/xen-legacy-backend.c | 254 +--- > hw/xen/xen-operations.c | 478 ++++++ > hw/xen/xen_devconfig.c | 4 +- > hw/xen/xen_pt.c | 2 +- > hw/xen/xen_pt.h | 2 +- > hw/xen/xen_pt_config_init.c | 2 +- > hw/xen/xen_pt_graphics.c | 1 - > hw/xen/xen_pt_msi.c | 4 +- > hw/xen/xen_pvdev.c | 63 +- > include/hw/xen/xen-bus-helper.h | 26 +- > include/hw/xen/xen-bus.h | 21 +- > include/hw/xen/xen-legacy-backend.h | 24 +- > include/hw/xen/xen.h | 24 +- > include/hw/xen/xen_backend_ops.h | 408 ++++++ > include/hw/xen/{xen_common.h => xen_native.h} | 75 +- > include/hw/xen/xen_pvdev.h | 6 +- > softmmu/globals.c | 4 + > target/i386/kvm/xen-emu.c | 5 + > tests/unit/meson.build | 1 + > tests/unit/test-xs-node.c | 871 +++++++++++ > 53 files changed, 5804 insertions(+), 927 deletions(-) > I think the series is good to go now so time to send a PR. Cheers, Paul
On Tue, 2023-03-07 at 17:57 +0000, Paul Durrant wrote:
> I think the series is good to go now so time to send a PR.
Done, thank you!
Following on from the basic platform support which has already been merged, here's phase 2 which wires up the XenBus and PV back ends. It starts with a basic single-tenant internal implementation of a XenStore, with a copy-on-write tree, watches, transactions, quotas. Then we introduce operations tables for the grant table, event channel, foreignmen and xenstore operations so that in addition to using the Xen libraries for those, QEMU can use its internal emulated versions. A little bit of cleaning up of header files, and we can enable the build of xen-bus in the CONFIG_XEN_EMU build, and run a Xen guest with an actual PV disk... qemu-system-x86_64 -serial mon:stdio -M q35 -display none -m 1G -smp 2 \ -accel kvm,xen-version=0x4000e,kernel-irqchip=split \ -kernel bzImage -append "console=ttyS0 root=/dev/xvda1 selinux=0" \ -drive file=/var/lib/libvirt/images/fedora28.qcow2,if=none,id=disk \ -device xen-disk,drive=disk,vdev=xvda The main thing that isn't working here is migration. I've implemented it for the internal xenstore and the unit tests exercise it, but the existing PV back ends don't support it, perhaps partly because support for guest transparent live migration support isn't upstream in Xen yet. So the disk doesn't come back correctly after migration. I'm content with that for 8.0 though, and we just mark the emulated XenStore device as unmigratable to prevent users from trying. The other pre-existing constraint is that only the block back end has yet been ported to the "new" XenBus infrastructure, and is actually capable of creating its own backend nodes. Again, I can live with that for 8.0. Maybe this will motivate us to finally get round to converting the rest off XenLegacyBackend and killing it. We also don't have a simple way to perform grant mapping of multiple guest pages to contiguous addresses, as we can under real Xen. So we don't advertise max-ring-page-order for xen-disk in the emulated mode. Fixing that — if we actually want to — would probably require mapping RAM from an actual backing store object, so that it can be mapped again at a different location for the PV back end to see. v2: https://git.infradead.org/users/dwmw2/qemu.git/shortlog/refs/heads/xenfv-2 • Full set of reviewed-by tags from Paul (and associated minor fixes). • Disable migration for emulated XenStore device. • Update docs and add MAINTAINERS entry. v1: https://lore.kernel.org/qemu-devel/20230302153435.1170111-1-dwmw2@infradead.org/ https://git.infradead.org/users/dwmw2/qemu.git/shortlog/refs/heads/xenfv-1 David Woodhouse (23): hw/xen: Add xenstore wire implementation and implementation stubs hw/xen: Add basic XenStore tree walk and write/read/directory support hw/xen: Implement XenStore watches hw/xen: Implement XenStore transactions hw/xen: Watches on XenStore transactions hw/xen: Implement core serialize/deserialize methods for xenstore_impl hw/xen: Add evtchn operations to allow redirection to internal emulation hw/xen: Add gnttab operations to allow redirection to internal emulation hw/xen: Pass grant ref to gnttab unmap operation hw/xen: Add foreignmem operations to allow redirection to internal emulation hw/xen: Move xenstore_store_pv_console_info to xen_console.c hw/xen: Use XEN_PAGE_SIZE in PV backend drivers hw/xen: Rename xen_common.h to xen_native.h hw/xen: Build PV backend drivers for CONFIG_XEN_BUS hw/xen: Only advertise ring-page-order for xen-block if gnttab supports it hw/xen: Hook up emulated implementation for event channel operations hw/xen: Add emulated implementation of grant table operations hw/xen: Add emulated implementation of XenStore operations hw/xen: Map guest XENSTORE_PFN grant in emulated Xenstore hw/xen: Implement soft reset for emulated gnttab i386/xen: Initialize Xen backends from pc_basic_device_init() for emulation MAINTAINERS: Add entry for Xen on KVM emulation docs: Update Xen-on-KVM documentation for PV disk support Paul Durrant (4): hw/xen: Implement XenStore permissions hw/xen: Create initial XenStore nodes hw/xen: Add xenstore operations to allow redirection to internal emulation hw/xen: Avoid crash when backend watch fires too early MAINTAINERS | 9 + accel/xen/xen-all.c | 69 +- docs/system/i386/xen.rst | 30 +- hw/9pfs/meson.build | 2 +- hw/9pfs/xen-9p-backend.c | 32 +- hw/block/dataplane/meson.build | 2 +- hw/block/dataplane/xen-block.c | 12 +- hw/block/meson.build | 2 +- hw/block/xen-block.c | 12 +- hw/char/meson.build | 2 +- hw/char/xen_console.c | 57 +- hw/display/meson.build | 2 +- hw/display/xenfb.c | 32 +- hw/i386/kvm/meson.build | 1 + hw/i386/kvm/trace-events | 15 + hw/i386/kvm/xen_evtchn.c | 15 + hw/i386/kvm/xen_gnttab.c | 325 ++++- hw/i386/kvm/xen_gnttab.h | 1 + hw/i386/kvm/xen_xenstore.c | 1251 +++++++++++++++- hw/i386/kvm/xenstore_impl.c | 1927 +++++++++++++++++++++++++ hw/i386/kvm/xenstore_impl.h | 63 + hw/i386/pc.c | 7 + hw/i386/pc_piix.c | 4 +- hw/i386/xen/xen-hvm.c | 38 +- hw/i386/xen/xen-mapcache.c | 2 +- hw/i386/xen/xen_platform.c | 7 +- hw/net/xen_nic.c | 25 +- hw/usb/meson.build | 2 +- hw/usb/xen-usb.c | 29 +- hw/xen/meson.build | 6 +- hw/xen/trace-events | 2 +- hw/xen/xen-bus-helper.c | 62 +- hw/xen/xen-bus.c | 411 +----- hw/xen/xen-legacy-backend.c | 254 +--- hw/xen/xen-operations.c | 478 ++++++ hw/xen/xen_devconfig.c | 4 +- hw/xen/xen_pt.c | 2 +- hw/xen/xen_pt.h | 2 +- hw/xen/xen_pt_config_init.c | 2 +- hw/xen/xen_pt_graphics.c | 1 - hw/xen/xen_pt_msi.c | 4 +- hw/xen/xen_pvdev.c | 63 +- include/hw/xen/xen-bus-helper.h | 26 +- include/hw/xen/xen-bus.h | 21 +- include/hw/xen/xen-legacy-backend.h | 24 +- include/hw/xen/xen.h | 24 +- include/hw/xen/xen_backend_ops.h | 408 ++++++ include/hw/xen/{xen_common.h => xen_native.h} | 75 +- include/hw/xen/xen_pvdev.h | 6 +- softmmu/globals.c | 4 + target/i386/kvm/xen-emu.c | 5 + tests/unit/meson.build | 1 + tests/unit/test-xs-node.c | 871 +++++++++++ 53 files changed, 5804 insertions(+), 927 deletions(-)