[v1,0/8] virtio-mem: Support "prealloc=on"

Message

David Hildenbrand Nov. 30, 2021, 10:41 a.m. UTC

Based-on: <[e-mail]>

Patch #1 - #7 are fully reviewed [1] but did not get picked up yet, so I'm
sending them along here, as they are required to use os_mem_prealloc() in
a safe way once the VM is running.

Support preallocation of memory to make virtio-mem safe to use with
scarce memory resources such as hugetlb. Before acknowledging a plug
request from the guest, we'll try preallcoating memory. If that fails,
we'll fail the request gracefully and warn the usr once.

To fully support huge pages for shared memory, we'll have to adjust shared
memory users, such as virtiofsd, to map guest memory via MAP_NORESERVE as
well, because otherwise, they'll end up overwriting the "reserve=off"
decision made by QEMU and try reserving huge pages for the sparse memory
region.

In the future, we might want to process guest requests, including
preallocating memory, asynchronously via a dedicated iothread.

[1] https://lkml.kernel.org/r/[e-mail]

Cc: Paolo Bonzini <[e-mail]>
Cc: "Michael S. Tsirkin" <[e-mail]>
Cc: Igor Mammedov <[e-mail]>
Cc: Eduardo Habkost <[e-mail]>
Cc: Dr. David Alan Gilbert <[e-mail]>
Cc: Daniel P. Berrangé <[e-mail]>
Cc: Gavin Shan <[e-mail]>
Cc: Hui Zhu <[e-mail]>
Cc: Sebastien Boeuf <[e-mail]>
Cc: Pankaj Gupta <[e-mail]>
Cc: Michal Prívozník <[e-mail]>

David Hildenbrand (8):
  util/oslib-posix: Let touch_all_pages() return an error
  util/oslib-posix: Support MADV_POPULATE_WRITE for os_mem_prealloc()
  util/oslib-posix: Introduce and use MemsetContext for
    touch_all_pages()
  util/oslib-posix: Don't create too many threads with small memory or
    little pages
  util/oslib-posix: Avoid creating a single thread with
    MADV_POPULATE_WRITE
  util/oslib-posix: Support concurrent os_mem_prealloc() invocation
  util/oslib-posix: Forward SIGBUS to MCE handler under Linux
  virtio-mem: Support "prealloc=on" option

 hw/virtio/virtio-mem.c         |  39 +++++-
 include/hw/virtio/virtio-mem.h |   4 +
 include/qemu/osdep.h           |   7 +
 softmmu/cpus.c                 |   4 +
 util/oslib-posix.c             | 231 +++++++++++++++++++++++++--------
 5 files changed, 226 insertions(+), 59 deletions(-)

Comments

Michal Prívozník Nov. 30, 2021, 4:32 p.m. UTC | #1

On 11/30/21 11:41, David Hildenbrand wrote:
> Based-on: <[e-mail]>
> 
> Patch #1 - #7 are fully reviewed [1] but did not get picked up yet, so I'm
> sending them along here, as they are required to use os_mem_prealloc() in
> a safe way once the VM is running.
> 
> Support preallocation of memory to make virtio-mem safe to use with
> scarce memory resources such as hugetlb. Before acknowledging a plug
> request from the guest, we'll try preallcoating memory. If that fails,
> we'll fail the request gracefully and warn the usr once.
> 
> To fully support huge pages for shared memory, we'll have to adjust shared
> memory users, such as virtiofsd, to map guest memory via MAP_NORESERVE as
> well, because otherwise, they'll end up overwriting the "reserve=off"
> decision made by QEMU and try reserving huge pages for the sparse memory
> region.
> 
> In the future, we might want to process guest requests, including
> preallocating memory, asynchronously via a dedicated iothread.
> 
> [1] https://lkml.kernel.org/r/[e-mail]
> 
> Cc: Paolo Bonzini <[e-mail]>
> Cc: "Michael S. Tsirkin" <[e-mail]>
> Cc: Igor Mammedov <[e-mail]>
> Cc: Eduardo Habkost <[e-mail]>
> Cc: Dr. David Alan Gilbert <[e-mail]>
> Cc: Daniel P. Berrangé <[e-mail]>
> Cc: Gavin Shan <[e-mail]>
> Cc: Hui Zhu <[e-mail]>
> Cc: Sebastien Boeuf <[e-mail]>
> Cc: Pankaj Gupta <[e-mail]>
> Cc: Michal Prívozník <[e-mail]>
> 
> David Hildenbrand (8):
>   util/oslib-posix: Let touch_all_pages() return an error
>   util/oslib-posix: Support MADV_POPULATE_WRITE for os_mem_prealloc()
>   util/oslib-posix: Introduce and use MemsetContext for
>     touch_all_pages()
>   util/oslib-posix: Don't create too many threads with small memory or
>     little pages
>   util/oslib-posix: Avoid creating a single thread with
>     MADV_POPULATE_WRITE
>   util/oslib-posix: Support concurrent os_mem_prealloc() invocation
>   util/oslib-posix: Forward SIGBUS to MCE handler under Linux
>   virtio-mem: Support "prealloc=on" option
> 
>  hw/virtio/virtio-mem.c         |  39 +++++-
>  include/hw/virtio/virtio-mem.h |   4 +
>  include/qemu/osdep.h           |   7 +
>  softmmu/cpus.c                 |   4 +
>  util/oslib-posix.c             | 231 +++++++++++++++++++++++++--------
>  5 files changed, 226 insertions(+), 59 deletions(-)
> 

Reviewed-by: Michal Privoznik <[e-mail]>

Michal