[v2,00/15] KVM: Dynamically size memslot arrays
mbox series

Message ID 20191022003537.13013-1-sean.j.christopherson@intel.com
Headers show
Series
  • KVM: Dynamically size memslot arrays
Related show

Message

Sean Christopherson Oct. 22, 2019, 12:35 a.m. UTC
The end goal of this series is to dynamically size the memslot array so
that KVM allocates memory based on the number of memslots in use, as
opposed to unconditionally allocating memory for the maximum number of
memslots.  On x86, each memslot consumes 88 bytes, and so with 2 address
spaces of 512 memslots, each VM consumes ~90k bytes for the memslots.
E.g. given a VM that uses a total of 30 memslots, dynamic sizing reduces
the memory footprint from 90k to ~2.6k bytes.

The changes required to support dynamic sizing are relatively small,
e.g. are essentially contained in patches 12/13 and 13/13.  Patches 1-11
clean up the memslot code, which has gotten quite crusy, especially
__kvm_set_memory_region().  The clean up is likely not strictly necessary
to switch to dynamic sizing, but I didn't have a remotely reasonable
level of confidence in the correctness of the dynamic sizing without first
doing the clean up.

Testing, especially non-x86 platforms, would be greatly appreciated.  The
non-x86 changes are for all intents and purposes untested, e.g. I compile
tested pieces of the code by copying them into x86, but that's it.  In
theory, the vast majority of the functional changes are arch agnostic, in
theory...

v2:
  - Split "Drop kvm_arch_create_memslot()" into three patches to move
    minor functional changes to standalone patches [Janosch].
  - Rebase to latest kvm/queue (f0574a1cea5b, "KVM: x86: fix ...")
  - Collect an Acked-by and a Reviewed-by

Sean Christopherson (15):
  KVM: Reinstall old memslots if arch preparation fails
  KVM: Don't free new memslot if allocation of said memslot fails
  KVM: PPC: Move memslot memory allocation into prepare_memory_region()
  KVM: x86: Allocate memslot resources during prepare_memory_region()
  KVM: Drop kvm_arch_create_memslot()
  KVM: Explicitly free allocated-but-unused dirty bitmap
  KVM: Refactor error handling for setting memory region
  KVM: Move setting of memslot into helper routine
  KVM: Move memslot deletion to helper function
  KVM: Simplify kvm_free_memslot() and all its descendents
  KVM: Clean up local variable usage in __kvm_set_memory_region()
  KVM: Provide common implementation for generic dirty log functions
  KVM: Ensure validity of memslot with respect to kvm_get_dirty_log()
  KVM: Terminate memslot walks via used_slots
  KVM: Dynamically size memslot array based on number of used slots

 arch/mips/include/asm/kvm_host.h      |   2 +-
 arch/mips/kvm/mips.c                  |  68 +---
 arch/powerpc/include/asm/kvm_ppc.h    |  14 +-
 arch/powerpc/kvm/book3s.c             |  22 +-
 arch/powerpc/kvm/book3s_hv.c          |  36 +-
 arch/powerpc/kvm/book3s_pr.c          |  20 +-
 arch/powerpc/kvm/booke.c              |  17 +-
 arch/powerpc/kvm/powerpc.c            |  13 +-
 arch/s390/include/asm/kvm_host.h      |   2 +-
 arch/s390/kvm/kvm-s390.c              |  21 +-
 arch/x86/include/asm/kvm_page_track.h |   3 +-
 arch/x86/kvm/page_track.c             |  15 +-
 arch/x86/kvm/x86.c                    | 100 ++---
 include/linux/kvm_host.h              |  48 +--
 virt/kvm/arm/arm.c                    |  47 +--
 virt/kvm/arm/mmu.c                    |  18 +-
 virt/kvm/kvm_main.c                   | 546 ++++++++++++++++----------
 17 files changed, 467 insertions(+), 525 deletions(-)

Comments

Paolo Bonzini Oct. 22, 2019, 1:59 p.m. UTC | #1
On 22/10/19 02:35, Sean Christopherson wrote:
> The end goal of this series is to dynamically size the memslot array so
> that KVM allocates memory based on the number of memslots in use, as
> opposed to unconditionally allocating memory for the maximum number of
> memslots.  On x86, each memslot consumes 88 bytes, and so with 2 address
> spaces of 512 memslots, each VM consumes ~90k bytes for the memslots.
> E.g. given a VM that uses a total of 30 memslots, dynamic sizing reduces
> the memory footprint from 90k to ~2.6k bytes.
> 
> The changes required to support dynamic sizing are relatively small,
> e.g. are essentially contained in patches 12/13 and 13/13.  Patches 1-11
> clean up the memslot code, which has gotten quite crusy, especially
> __kvm_set_memory_region().  The clean up is likely not strictly necessary
> to switch to dynamic sizing, but I didn't have a remotely reasonable
> level of confidence in the correctness of the dynamic sizing without first
> doing the clean up.
> 
> Testing, especially non-x86 platforms, would be greatly appreciated.  The
> non-x86 changes are for all intents and purposes untested, e.g. I compile
> tested pieces of the code by copying them into x86, but that's it.  In
> theory, the vast majority of the functional changes are arch agnostic, in
> theory...
> 
> v2:
>   - Split "Drop kvm_arch_create_memslot()" into three patches to move
>     minor functional changes to standalone patches [Janosch].
>   - Rebase to latest kvm/queue (f0574a1cea5b, "KVM: x86: fix ...")
>   - Collect an Acked-by and a Reviewed-by
> 
> Sean Christopherson (15):
>   KVM: Reinstall old memslots if arch preparation fails
>   KVM: Don't free new memslot if allocation of said memslot fails
>   KVM: PPC: Move memslot memory allocation into prepare_memory_region()
>   KVM: x86: Allocate memslot resources during prepare_memory_region()
>   KVM: Drop kvm_arch_create_memslot()
>   KVM: Explicitly free allocated-but-unused dirty bitmap
>   KVM: Refactor error handling for setting memory region
>   KVM: Move setting of memslot into helper routine
>   KVM: Move memslot deletion to helper function
>   KVM: Simplify kvm_free_memslot() and all its descendents
>   KVM: Clean up local variable usage in __kvm_set_memory_region()
>   KVM: Provide common implementation for generic dirty log functions
>   KVM: Ensure validity of memslot with respect to kvm_get_dirty_log()
>   KVM: Terminate memslot walks via used_slots
>   KVM: Dynamically size memslot array based on number of used slots
> 
>  arch/mips/include/asm/kvm_host.h      |   2 +-
>  arch/mips/kvm/mips.c                  |  68 +---
>  arch/powerpc/include/asm/kvm_ppc.h    |  14 +-
>  arch/powerpc/kvm/book3s.c             |  22 +-
>  arch/powerpc/kvm/book3s_hv.c          |  36 +-
>  arch/powerpc/kvm/book3s_pr.c          |  20 +-
>  arch/powerpc/kvm/booke.c              |  17 +-
>  arch/powerpc/kvm/powerpc.c            |  13 +-
>  arch/s390/include/asm/kvm_host.h      |   2 +-
>  arch/s390/kvm/kvm-s390.c              |  21 +-
>  arch/x86/include/asm/kvm_page_track.h |   3 +-
>  arch/x86/kvm/page_track.c             |  15 +-
>  arch/x86/kvm/x86.c                    | 100 ++---
>  include/linux/kvm_host.h              |  48 +--
>  virt/kvm/arm/arm.c                    |  47 +--
>  virt/kvm/arm/mmu.c                    |  18 +-
>  virt/kvm/kvm_main.c                   | 546 ++++++++++++++++----------
>  17 files changed, 467 insertions(+), 525 deletions(-)
> 

Christian, Marc, Paul, can you help testing patches 1-13?

Thanks,
Paolo Bonzini Oct. 22, 2019, 2:04 p.m. UTC | #2
On 22/10/19 02:35, Sean Christopherson wrote:
> The end goal of this series is to dynamically size the memslot array so
> that KVM allocates memory based on the number of memslots in use, as
> opposed to unconditionally allocating memory for the maximum number of
> memslots.  On x86, each memslot consumes 88 bytes, and so with 2 address
> spaces of 512 memslots, each VM consumes ~90k bytes for the memslots.
> E.g. given a VM that uses a total of 30 memslots, dynamic sizing reduces
> the memory footprint from 90k to ~2.6k bytes.
> 
> The changes required to support dynamic sizing are relatively small,
> e.g. are essentially contained in patches 12/13 and 13/13.  Patches 1-11
> clean up the memslot code, which has gotten quite crusy, especially
> __kvm_set_memory_region().  The clean up is likely not strictly necessary
> to switch to dynamic sizing, but I didn't have a remotely reasonable
> level of confidence in the correctness of the dynamic sizing without first
> doing the clean up.
> 
> Testing, especially non-x86 platforms, would be greatly appreciated.  The
> non-x86 changes are for all intents and purposes untested, e.g. I compile
> tested pieces of the code by copying them into x86, but that's it.  In
> theory, the vast majority of the functional changes are arch agnostic, in
> theory...
> 
> v2:
>   - Split "Drop kvm_arch_create_memslot()" into three patches to move
>     minor functional changes to standalone patches [Janosch].
>   - Rebase to latest kvm/queue (f0574a1cea5b, "KVM: x86: fix ...")
>   - Collect an Acked-by and a Reviewed-by

I only have some cosmetic changes on patches 14-15.  Let's wait for
testing results.

Paolo
Christoffer Dall Oct. 23, 2019, 9:39 a.m. UTC | #3
On Mon, Oct 21, 2019 at 05:35:22PM -0700, Sean Christopherson wrote:
> The end goal of this series is to dynamically size the memslot array so
> that KVM allocates memory based on the number of memslots in use, as
> opposed to unconditionally allocating memory for the maximum number of
> memslots.  On x86, each memslot consumes 88 bytes, and so with 2 address
> spaces of 512 memslots, each VM consumes ~90k bytes for the memslots.
> E.g. given a VM that uses a total of 30 memslots, dynamic sizing reduces
> the memory footprint from 90k to ~2.6k bytes.
> 
> The changes required to support dynamic sizing are relatively small,
> e.g. are essentially contained in patches 12/13 and 13/13.  Patches 1-11
> clean up the memslot code, which has gotten quite crusy, especially
> __kvm_set_memory_region().  The clean up is likely not strictly necessary
> to switch to dynamic sizing, but I didn't have a remotely reasonable
> level of confidence in the correctness of the dynamic sizing without first
> doing the clean up.
> 
> Testing, especially non-x86 platforms, would be greatly appreciated.  The
> non-x86 changes are for all intents and purposes untested, e.g. I compile
> tested pieces of the code by copying them into x86, but that's it.  In
> theory, the vast majority of the functional changes are arch agnostic, in
> theory...

I've built this for arm/arm64, and I've ran my usual set of tests which
pass fine.  I've also run the selftest framework's tests for the dirty
logging and the migration loop test for arm64, and they pass fine.

You can add my (for arm64):

Tested-by: Christoffer Dall <christoffer.dall@arm.com>
Christian Borntraeger Oct. 23, 2019, 6:56 p.m. UTC | #4
On 22.10.19 15:59, Paolo Bonzini wrote:
> On 22/10/19 02:35, Sean Christopherson wrote:
>> The end goal of this series is to dynamically size the memslot array so
>> that KVM allocates memory based on the number of memslots in use, as
>> opposed to unconditionally allocating memory for the maximum number of
>> memslots.  On x86, each memslot consumes 88 bytes, and so with 2 address
>> spaces of 512 memslots, each VM consumes ~90k bytes for the memslots.
>> E.g. given a VM that uses a total of 30 memslots, dynamic sizing reduces
>> the memory footprint from 90k to ~2.6k bytes.
>>
>> The changes required to support dynamic sizing are relatively small,
>> e.g. are essentially contained in patches 12/13 and 13/13.  Patches 1-11
>> clean up the memslot code, which has gotten quite crusy, especially
>> __kvm_set_memory_region().  The clean up is likely not strictly necessary
>> to switch to dynamic sizing, but I didn't have a remotely reasonable
>> level of confidence in the correctness of the dynamic sizing without first
>> doing the clean up.
>>
>> Testing, especially non-x86 platforms, would be greatly appreciated.  The
>> non-x86 changes are for all intents and purposes untested, e.g. I compile
>> tested pieces of the code by copying them into x86, but that's it.  In
>> theory, the vast majority of the functional changes are arch agnostic, in
>> theory...
>>
>> v2:
>>   - Split "Drop kvm_arch_create_memslot()" into three patches to move
>>     minor functional changes to standalone patches [Janosch].
>>   - Rebase to latest kvm/queue (f0574a1cea5b, "KVM: x86: fix ...")
>>   - Collect an Acked-by and a Reviewed-by
>>
>> Sean Christopherson (15):
>>   KVM: Reinstall old memslots if arch preparation fails
>>   KVM: Don't free new memslot if allocation of said memslot fails
>>   KVM: PPC: Move memslot memory allocation into prepare_memory_region()
>>   KVM: x86: Allocate memslot resources during prepare_memory_region()
>>   KVM: Drop kvm_arch_create_memslot()
>>   KVM: Explicitly free allocated-but-unused dirty bitmap
>>   KVM: Refactor error handling for setting memory region
>>   KVM: Move setting of memslot into helper routine
>>   KVM: Move memslot deletion to helper function
>>   KVM: Simplify kvm_free_memslot() and all its descendents
>>   KVM: Clean up local variable usage in __kvm_set_memory_region()
>>   KVM: Provide common implementation for generic dirty log functions
>>   KVM: Ensure validity of memslot with respect to kvm_get_dirty_log()
>>   KVM: Terminate memslot walks via used_slots
>>   KVM: Dynamically size memslot array based on number of used slots
>>
>>  arch/mips/include/asm/kvm_host.h      |   2 +-
>>  arch/mips/kvm/mips.c                  |  68 +---
>>  arch/powerpc/include/asm/kvm_ppc.h    |  14 +-
>>  arch/powerpc/kvm/book3s.c             |  22 +-
>>  arch/powerpc/kvm/book3s_hv.c          |  36 +-
>>  arch/powerpc/kvm/book3s_pr.c          |  20 +-
>>  arch/powerpc/kvm/booke.c              |  17 +-
>>  arch/powerpc/kvm/powerpc.c            |  13 +-
>>  arch/s390/include/asm/kvm_host.h      |   2 +-
>>  arch/s390/kvm/kvm-s390.c              |  21 +-
>>  arch/x86/include/asm/kvm_page_track.h |   3 +-
>>  arch/x86/kvm/page_track.c             |  15 +-
>>  arch/x86/kvm/x86.c                    | 100 ++---
>>  include/linux/kvm_host.h              |  48 +--
>>  virt/kvm/arm/arm.c                    |  47 +--
>>  virt/kvm/arm/mmu.c                    |  18 +-
>>  virt/kvm/kvm_main.c                   | 546 ++++++++++++++++----------
>>  17 files changed, 467 insertions(+), 525 deletions(-)
>>
> 
> Christian, Marc, Paul, can you help testing patches 1-13?

I see no obvious breakage on s390, but I will do some more testing in the next days.