| Message ID | 20191018105315.27511-1-slp@redhat.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Introduce the microvm machine type | expand |
Sergio Lopez <slp@redhat.com> writes: > microvm is a machine type inspired by Firecracker and constructed > after its machine model. > > It's a minimalist machine type without PCI nor ACPI support, designed > for short-lived guests. microvm also establishes a baseline for > benchmarking and optimizing both QEMU and guest operating systems, > since it is optimized for both boot time and footprint. I know this has been merged, but I ask anyway: got some boot time and memory footprint measurements?
Markus Armbruster <armbru@redhat.com> writes: > Sergio Lopez <slp@redhat.com> writes: > >> microvm is a machine type inspired by Firecracker and constructed >> after its machine model. >> >> It's a minimalist machine type without PCI nor ACPI support, designed >> for short-lived guests. microvm also establishes a baseline for >> benchmarking and optimizing both QEMU and guest operating systems, >> since it is optimized for both boot time and footprint. > > I know this has been merged, but I ask anyway: got some boot time and > memory footprint measurements? It's been a while since I ran an instrumented kernel, but I've been playing a bit with OSv (thanks to a question in qemu-discuss), which is aggressively optimized for boot time, and I have some numbers. This is using a QEMU build with some patches to be able to do a microvm-only build that "I have not yet submitted but shouldn't be controversial" (TM) [1]. The Q35 binary was also built having only the Q35 machine type enabled. In this test, OSv runs to userspace and then shuts down. OSv gives us its boot time (from entry point to userspace), and I also get the execution totals with "time". Variation between runs is +-2%: ------------ | qemu-q35 | ------------ Boot time ========= time ./x86_64-softmmu/qemu-system-x86_64 -M q35 -enable-kvm \ -cpu host -m 64m -kernel ~/osv/build/release/loader-stripped.elf \ -append "/hello" -serial stdio -nodefaults -no-user-config \ -nographic -no-reboot -device virtio-blk,id=blk0,drive=hd0,scsi=off \ -drive file=~/osv/build/release/usr.img,if=none,id=hd0,cache=none \ -global virtio-mmio.force-legacy=off -no-reboot SeaBIOS (version rel-1.12.0-59-gc9ba5276e321-prebuilt.qemu.org) Booting from ROM..OSv v0.54.0-26-gcccc7d40 Booted up in 11.77 ms Cmdline: /hello Hello from C code real 0m0.073s user 0m0.034s sys 0m0.029s Size ==== # ls -l x86_64-softmmu/qemu-system-x86_64* -rwxr-xr-x. 1 root root 13400680 Dec 2 07:31 x86_64-softmmu/qemu-system-x86_64 -rwxr-xr-x. 1 root root 11707344 Dec 2 07:51 x86_64-softmmu/qemu-system-x86_64.stripped ---------------- | qemu-microvm | ---------------- Boot time ========= time ./x86_64-softmmu/qemu-system-x86_64 -M microvm -enable-kvm \ -cpu host -m 64m -smp 1 \ -kernel ~/osv/build/release/loader-stripped.elf \ -append "--nopci /hello" -serial stdio -nodefaults -no-user-config \ -nographic -no-reboot \ -device virtio-blk-device,id=blk0,drive=hd0,scsi=off \ -drive file=~/osv/build/release/usr.img,if=none,id=hd0 \ -global virtio-mmio.force-legacy=off OSv v0.54.0-26-gcccc7d40 Booted up in 3.64 ms Cmdline: /hello Hello from C code real 0m0.020s user 0m0.009s sys 0m0.011s Size ==== # ls -l x86_64-softmmu/qemu-system-x86_64* -rwxr-xr-x. 1 root root 7823344 Dec 2 07:35 x86_64-softmmu/qemu-system-x86_64 -rwxr-xr-x. 1 root root 6486992 Dec 2 07:55 x86_64-softmmu/qemu-system-x86_64.stripped I don't have numbers for memory footprint. It'd be nice coming up with some reasonable methodology for getting them, as there a significant amount of moving parts there that can have an impact on memory consumption. Sergio. [1] https://github.com/slp/qemu/tree/microvm_only
microvm is a machine type inspired by Firecracker and constructed after its machine model. It's a minimalist machine type without PCI nor ACPI support, designed for short-lived guests. microvm also establishes a baseline for benchmarking and optimizing both QEMU and guest operating systems, since it is optimized for both boot time and footprint. --- Changelog v11: - Remove leftovers from pre-v3 series in microvm.h (Marc-André Lureau) - Add a "Limitations" section to the documentation (Daniel P. Berrangé) - Remove "server" flag from an example in the documentation (Marc-André Lureau) v10: - Fix typo in documentation and cover (Greg Kurz) - Fix commit message in patch 04/14 (Philippe Mathieu-Daudé) - Ensure "microvm" appears in lowercase everywhere except in MachineClass and MachineState struct derivatives (Philippe Mathieu-Daudé) - Add missing components to MICROVM config in Kconfig (Paolo Bonzini, Philippe Mathieu-Daudé) v9: - Fix a typo in "[PATCH v9 05/15] hw/i386/pc: avoid an assignment in if condition in x86_load_linux()" (Philippe Mathieu-Daudé) - Replace qemu_strtol() with qemu_strtoui() to preserve the original type of video_mode (Philippe Mathieu-Daudé) v8: - Split "[PATCH v7 03/12] hw/i386/pc: fix code style issues on functions that will be moved out" into four different patches (Philippe Mathieu-Daudé) v7: - Fix code style issues on already present code touched by this patch series (Michael S. Tsirkin, Philippe Mathieu-Daudé) - Add new files to MAINTAINERS (Michael S. Tsirkin, Philippe Mathieu-Daudé) - Allow starting a microvm machine without a kernel image, fixing "qom-test" (Michael S. Tsirkin) - Change "bios-microvm.bin" mode to 0644 (Stefano Garzarella) - Remove unneeded "hw/i386/pc.h" include from x86.c (Stefano Garzarella) v6: - Some style fixes (Philippe Mathieu-Daudé) - Fix a documentation bug stating that LAPIC was in userspace (Paolo Bonzini) - Update Xen HVM code after X86MachineState introduction (Philippe Mathieu-Daudé) - Rename header guard from QEMU_VIRTIO_MMIO_H to HW_VIRTIO_MMIO_H (Philippe Mathieu-Daudé) v5: - Drop unneeded "[PATCH v4 2/8] hw/i386: Factorize e820 related functions" (Philippe Mathieu-Daudé) - Drop unneeded "[PATCH v4 1/8] hw/i386: Factorize PVH related functions" (Stefano Garzarella) - Split X86MachineState introduction into smaller patches (Philippe Mathieu-Daudé) - Change option-roms to x-option-roms and kernel-cmdline to auto-kernel-cmdline (Paolo Bonzini) - Make i8259 PIT and i8254 PIC optional (Paolo Bonzini) - Some fixes to the documentation (Paolo Bonzini) - Switch documentation format from txt to rst (Peter Maydell) - Move NMI interface to X86_MACHINE (Philippe Mathieu-Daudé, Paolo Bonzini) v4: - This is a complete rewrite of the whole patchset, with a focus on reusing as much existing code as possible to ease the maintenance burden and making the machine type as compatible as possible by default. As a result, the number of lines dedicated specifically to microvm is 383 (code lines measured by "cloc") and, with the default configuration, it's now able to boot both PVH ELF images and bzImages with either SeaBIOS or qboot. v3: - Add initrd support (thanks Stefano). v2: - Drop "[PATCH 1/4] hw/i386: Factorize CPU routine". - Simplify machine definition (thanks Eduardo). - Remove use of unneeded NUMA-related callbacks (thanks Eduardo). - Add a patch to factorize PVH-related functions. - Replace use of Linux's Zero Page with PVH (thanks Maran and Paolo). --- Sergio Lopez (15): hw/virtio: Factorize virtio-mmio headers hw/i386/pc: rename functions shared with non-PC machines hw/i386/pc: fix code style issues on functions that will be moved out hw/i386/pc: replace use of strtol with qemu_strtoui in x86_load_linux() hw/i386/pc: avoid an assignment in if condition in x86_load_linux() hw/i386/pc: remove commented out code from x86_load_linux() hw/i386/pc: move shared x86 functions to x86.c and export them hw/i386: split PCMachineState deriving X86MachineState from it hw/i386: make x86.c independent from PCMachineState fw_cfg: add "modify" functions for all types hw/intc/apic: reject pic ints if isa_pic == NULL roms: add microvm-bios (qboot) as binary and git submodule docs/microvm.rst: document the new microvm machine type hw/i386: Introduce the microvm machine type MAINTAINERS: add microvm related files docs/microvm.rst | 108 +++++ default-configs/i386-softmmu.mak | 1 + include/hw/i386/microvm.h | 71 +++ include/hw/i386/pc.h | 28 +- include/hw/i386/x86.h | 96 ++++ include/hw/nvram/fw_cfg.h | 42 ++ include/hw/virtio/virtio-mmio.h | 73 +++ hw/acpi/cpu_hotplug.c | 10 +- hw/i386/acpi-build.c | 29 +- hw/i386/amd_iommu.c | 3 +- hw/i386/intel_iommu.c | 3 +- hw/i386/microvm.c | 572 ++++++++++++++++++++++ hw/i386/pc.c | 781 +++--------------------------- hw/i386/pc_piix.c | 46 +- hw/i386/pc_q35.c | 38 +- hw/i386/pc_sysfw.c | 60 +-- hw/i386/x86.c | 795 +++++++++++++++++++++++++++++++ hw/i386/xen/xen-hvm.c | 28 +- hw/intc/apic.c | 2 +- hw/intc/ioapic.c | 2 +- hw/nvram/fw_cfg.c | 29 ++ hw/virtio/virtio-mmio.c | 48 +- .gitmodules | 3 + MAINTAINERS | 9 + hw/i386/Kconfig | 10 + hw/i386/Makefile.objs | 2 + pc-bios/bios-microvm.bin | Bin 0 -> 65536 bytes roms/Makefile | 6 + roms/qboot | 1 + 29 files changed, 1985 insertions(+), 911 deletions(-) create mode 100644 docs/microvm.rst create mode 100644 include/hw/i386/microvm.h create mode 100644 include/hw/i386/x86.h create mode 100644 include/hw/virtio/virtio-mmio.h create mode 100644 hw/i386/microvm.c create mode 100644 hw/i386/x86.c create mode 100644 pc-bios/bios-microvm.bin create mode 160000 roms/qboot