[v1,0/5] virtio-mem: s390x support

Message

David Hildenbrand Sept. 10, 2024, 7:15 p.m. UTC

Let's finally add s390x support for virtio-mem; my last RFC was sent
4 years ago, and a lot changed in the meantime.

This is based on mm/stable.

I sent out the QEMU part earlier today [1], that contains some more details
and a usage example on s390x (last patch).

There is not too much in here: The biggest part is querying a new diag(500)
STORAGE_LIMIT hypercall to obtain the proper "max_physmem_end". Once this
and the QEMU part will go upstream, it will get documented in [2]

The last two patches are not strictly required but certainly nice-to-have.

Note that -- in contrast to standby memory -- virtio-mem memory must be
configured to be automatically onlined as soon as hotplugged. The easiest
approach is using the "memhp_default_state=" kernel parameter or by using
proper udev rules. More details can be found at [3].

I have reviving+upstreaming a systemd service to handle configuring
that on my todo list, but for some reason I keep getting distracted ...

I tested various things, including:
 * Various memory hotplug/hotunplug combinations
 * Device hotplug/hotunplug
 * /proc/iomem output
 * reboot
 * kexec
 * kdump: make sure we don't hotplug memory

One remaining work item is kdump support for virtio-mem memory. I
am working on a prototype that will be fairly straight forward,
because the virtio-mem driver already supports a special kdump mode and
dracut will already include it in the initrd as default. With
holiday and conferences coming up I rather sent this out now.

[1] https://lkml.kernel.org/r/[e-mail]
[2] https://gitlab.com/davidhildenbrand/s390x-os-virt-spec
[3] https://virtio-mem.gitlab.io/user-guide/user-guide-linux.html

Cc: Heiko Carstens <[e-mail]>
Cc: Vasily Gorbik <[e-mail]>
Cc: Alexander Gordeev <[e-mail]>
Cc: Christian Borntraeger <[e-mail]>
Cc: Sven Schnelle <[e-mail]>
Cc: Thomas Huth <[e-mail]>
Cc: Cornelia Huck <[e-mail]>
Cc: Janosch Frank <[e-mail]>
Cc: Claudio Imbrenda <[e-mail]>
Cc: "Michael S. Tsirkin" <[e-mail]>
Cc: Jason Wang <[e-mail]>
Cc: Xuan Zhuo <[e-mail]>
Cc: "Eugenio Pérez" <[e-mail]>
Cc: Andrew Morton <[e-mail]>

David Hildenbrand (5):
  s390/kdump: implement is_kdump_kernel()
  s390/physmem_info: query diag500(STORAGE_LIMIT) to support QEMU/KVM
    memory devices
  virtio-mem: s390x support
  lib/Kconfig.debug: default STRICT_DEVMEM to "y" on s390x
  s390/sparsemem: reduce section size to 128 MiB

 arch/s390/boot/physmem_info.c        | 46 ++++++++++++++++++++++++++--
 arch/s390/include/asm/kexec.h        |  4 +++
 arch/s390/include/asm/physmem_info.h |  3 ++
 arch/s390/include/asm/sparsemem.h    |  2 +-
 arch/s390/kernel/crash_dump.c        |  6 ++++
 drivers/virtio/Kconfig               | 12 ++++----
 lib/Kconfig.debug                    |  2 +-
 7 files changed, 64 insertions(+), 11 deletions(-)

Comments

Michael S. Tsirkin Sept. 10, 2024, 8:19 p.m. UTC | #1

On Tue, Sep 10, 2024 at 09:15:34PM +0200, David Hildenbrand wrote:
> Let's finally add s390x support for virtio-mem; my last RFC was sent
> 4 years ago, and a lot changed in the meantime.
> 
> This is based on mm/stable.
> 
> I sent out the QEMU part earlier today [1], that contains some more details
> and a usage example on s390x (last patch).
> 
> There is not too much in here: The biggest part is querying a new diag(500)
> STORAGE_LIMIT hypercall to obtain the proper "max_physmem_end". Once this
> and the QEMU part will go upstream, it will get documented in [2]
> 
> The last two patches are not strictly required but certainly nice-to-have.
> 
> Note that -- in contrast to standby memory -- virtio-mem memory must be
> configured to be automatically onlined as soon as hotplugged. The easiest
> approach is using the "memhp_default_state=" kernel parameter or by using
> proper udev rules. More details can be found at [3].
> 
> I have reviving+upstreaming a systemd service to handle configuring
> that on my todo list, but for some reason I keep getting distracted ...
> 
> I tested various things, including:
>  * Various memory hotplug/hotunplug combinations
>  * Device hotplug/hotunplug
>  * /proc/iomem output
>  * reboot
>  * kexec
>  * kdump: make sure we don't hotplug memory
> 
> One remaining work item is kdump support for virtio-mem memory. I
> am working on a prototype that will be fairly straight forward,
> because the virtio-mem driver already supports a special kdump mode and
> dracut will already include it in the initrd as default. With
> holiday and conferences coming up I rather sent this out now.
> 
> [1] https://lkml.kernel.org/r/[e-mail]
> [2] https://gitlab.com/davidhildenbrand/s390x-os-virt-spec
> [3] https://virtio-mem.gitlab.io/user-guide/user-guide-linux.html
> 
> Cc: Heiko Carstens <[e-mail]>
> Cc: Vasily Gorbik <[e-mail]>
> Cc: Alexander Gordeev <[e-mail]>
> Cc: Christian Borntraeger <[e-mail]>
> Cc: Sven Schnelle <[e-mail]>
> Cc: Thomas Huth <[e-mail]>
> Cc: Cornelia Huck <[e-mail]>
> Cc: Janosch Frank <[e-mail]>
> Cc: Claudio Imbrenda <[e-mail]>
> Cc: "Michael S. Tsirkin" <[e-mail]>
> Cc: Jason Wang <[e-mail]>
> Cc: Xuan Zhuo <[e-mail]>
> Cc: "Eugenio Pérez" <[e-mail]>
> Cc: Andrew Morton <[e-mail]>


It's mostly s390 changes, so should be merged through that tree.
I acked the only virtio specific patch.

> David Hildenbrand (5):
>   s390/kdump: implement is_kdump_kernel()
>   s390/physmem_info: query diag500(STORAGE_LIMIT) to support QEMU/KVM
>     memory devices
>   virtio-mem: s390x support
>   lib/Kconfig.debug: default STRICT_DEVMEM to "y" on s390x
>   s390/sparsemem: reduce section size to 128 MiB
> 
>  arch/s390/boot/physmem_info.c        | 46 ++++++++++++++++++++++++++--
>  arch/s390/include/asm/kexec.h        |  4 +++
>  arch/s390/include/asm/physmem_info.h |  3 ++
>  arch/s390/include/asm/sparsemem.h    |  2 +-
>  arch/s390/kernel/crash_dump.c        |  6 ++++
>  drivers/virtio/Kconfig               | 12 ++++----
>  lib/Kconfig.debug                    |  2 +-
>  7 files changed, 64 insertions(+), 11 deletions(-)
> 
> -- 
> 2.46.0