mbox series

[v3,00/15] KVM: Dynamically size memslot arrays

Message ID 20191024230744.14543-1-sean.j.christopherson@intel.com (mailing list archive)
Headers show
Series KVM: Dynamically size memslot arrays | expand

Message

Sean Christopherson Oct. 24, 2019, 11:07 p.m. UTC
The end goal of this series is to dynamically size the memslot array so
that KVM allocates memory based on the number of memslots in use, as
opposed to unconditionally allocating memory for the maximum number of
memslots.  On x86, each memslot consumes 88 bytes, and so with 2 address
spaces of 512 memslots, each VM consumes ~90k bytes for the memslots.
E.g. given a VM that uses a total of 30 memslots, dynamic sizing reduces
the memory footprint from 90k to ~2.6k bytes.

The changes required to support dynamic sizing are relatively small,
e.g. are essentially contained in patches 14/15 and 15/15.  Patches 1-13
clean up the memslot code, which has gotten quite crusty, especially
__kvm_set_memory_region().  The clean up is likely not strictly necessary
to switch to dynamic sizing, but I didn't have a remotely reasonable
level of confidence in the correctness of the dynamic sizing without first
doing the clean up.

Christoffer, I added your Tested-by to the patches that I was confident
would be fully tested based on the desription of what you tested.  Let me
know if you disagree with any of 'em.

v3:
  - Fix build errors on PPC and MIPS due to missed params during
    refactoring [kbuild test robot].
  - Rename the helpers for update_memslots() and add comments describing
    the new algorithm and how it interacts with searching [Paolo].
  - Remove the unnecessary and obnoxious warning regarding memslots being
    a flexible array [Paolo].
  - Fix typos in the changelog of patch 09/15 [Christoffer].
  - Collect tags [Christoffer].

v2:
  - Split "Drop kvm_arch_create_memslot()" into three patches to move
    minor functional changes to standalone patches [Janosch].
  - Rebase to latest kvm/queue (f0574a1cea5b, "KVM: x86: fix ...")
  - Collect an Acked-by and a Reviewed-by


Sean Christopherson (15):
  KVM: Reinstall old memslots if arch preparation fails
  KVM: Don't free new memslot if allocation of said memslot fails
  KVM: PPC: Move memslot memory allocation into prepare_memory_region()
  KVM: x86: Allocate memslot resources during prepare_memory_region()
  KVM: Drop kvm_arch_create_memslot()
  KVM: Explicitly free allocated-but-unused dirty bitmap
  KVM: Refactor error handling for setting memory region
  KVM: Move setting of memslot into helper routine
  KVM: Move memslot deletion to helper function
  KVM: Simplify kvm_free_memslot() and all its descendents
  KVM: Clean up local variable usage in __kvm_set_memory_region()
  KVM: Provide common implementation for generic dirty log functions
  KVM: Ensure validity of memslot with respect to kvm_get_dirty_log()
  KVM: Terminate memslot walks via used_slots
  KVM: Dynamically size memslot array based on number of used slots

 arch/mips/include/asm/kvm_host.h      |   2 +-
 arch/mips/kvm/mips.c                  |  69 +--
 arch/powerpc/include/asm/kvm_ppc.h    |  17 +-
 arch/powerpc/kvm/book3s.c             |  22 +-
 arch/powerpc/kvm/book3s_hv.c          |  36 +-
 arch/powerpc/kvm/book3s_pr.c          |  20 +-
 arch/powerpc/kvm/booke.c              |  17 +-
 arch/powerpc/kvm/powerpc.c            |  13 +-
 arch/s390/include/asm/kvm_host.h      |   2 +-
 arch/s390/kvm/kvm-s390.c              |  21 +-
 arch/x86/include/asm/kvm_page_track.h |   3 +-
 arch/x86/kvm/page_track.c             |  15 +-
 arch/x86/kvm/x86.c                    | 101 +----
 include/linux/kvm_host.h              |  46 +-
 virt/kvm/arm/arm.c                    |  48 +-
 virt/kvm/arm/mmu.c                    |  18 +-
 virt/kvm/kvm_main.c                   | 621 +++++++++++++++++---------
 17 files changed, 542 insertions(+), 529 deletions(-)

Comments

Christoffer Dall Oct. 25, 2019, 1:28 p.m. UTC | #1
On Thu, Oct 24, 2019 at 04:07:29PM -0700, Sean Christopherson wrote:
> The end goal of this series is to dynamically size the memslot array so
> that KVM allocates memory based on the number of memslots in use, as
> opposed to unconditionally allocating memory for the maximum number of
> memslots.  On x86, each memslot consumes 88 bytes, and so with 2 address
> spaces of 512 memslots, each VM consumes ~90k bytes for the memslots.
> E.g. given a VM that uses a total of 30 memslots, dynamic sizing reduces
> the memory footprint from 90k to ~2.6k bytes.
> 
> The changes required to support dynamic sizing are relatively small,
> e.g. are essentially contained in patches 14/15 and 15/15.  Patches 1-13
> clean up the memslot code, which has gotten quite crusty, especially
> __kvm_set_memory_region().  The clean up is likely not strictly necessary
> to switch to dynamic sizing, but I didn't have a remotely reasonable
> level of confidence in the correctness of the dynamic sizing without first
> doing the clean up.
> 
> Christoffer, I added your Tested-by to the patches that I was confident
> would be fully tested based on the desription of what you tested.  Let me
> know if you disagree with any of 'em.
> 
The only testing I've done of patch 9 would be via the vm_free part of
kvm selftest, so not sure how valid that is, but sure.

Looks fine otherwise.


Thanks,

    Christoffer
Marc Zyngier Oct. 25, 2019, 4:30 p.m. UTC | #2
On 2019-10-25 00:07, Sean Christopherson wrote:
> The end goal of this series is to dynamically size the memslot array 
> so
> that KVM allocates memory based on the number of memslots in use, as
> opposed to unconditionally allocating memory for the maximum number 
> of
> memslots.  On x86, each memslot consumes 88 bytes, and so with 2 
> address
> spaces of 512 memslots, each VM consumes ~90k bytes for the memslots.
> E.g. given a VM that uses a total of 30 memslots, dynamic sizing 
> reduces
> the memory footprint from 90k to ~2.6k bytes.
>
> The changes required to support dynamic sizing are relatively small,
> e.g. are essentially contained in patches 14/15 and 15/15.  Patches 
> 1-13
> clean up the memslot code, which has gotten quite crusty, especially
> __kvm_set_memory_region().  The clean up is likely not strictly 
> necessary
> to switch to dynamic sizing, but I didn't have a remotely reasonable
> level of confidence in the correctness of the dynamic sizing without 
> first
> doing the clean up.

I've finally found time to test this on a garden variety of arm64 
boxes,
and nothing caught fire. It surely must be doing something right!

FWIW:

Tested-by: Marc Zyngier <maz@kernel.org>

         M.
Sean Christopherson Dec. 3, 2019, 10:14 p.m. UTC | #3
On Thu, Oct 24, 2019 at 04:07:29PM -0700, Sean Christopherson wrote:
> The end goal of this series is to dynamically size the memslot array so
> that KVM allocates memory based on the number of memslots in use, as
> opposed to unconditionally allocating memory for the maximum number of
> memslots.  On x86, each memslot consumes 88 bytes, and so with 2 address
> spaces of 512 memslots, each VM consumes ~90k bytes for the memslots.
> E.g. given a VM that uses a total of 30 memslots, dynamic sizing reduces
> the memory footprint from 90k to ~2.6k bytes.
> 
> The changes required to support dynamic sizing are relatively small,
> e.g. are essentially contained in patches 14/15 and 15/15.  Patches 1-13
> clean up the memslot code, which has gotten quite crusty, especially
> __kvm_set_memory_region().  The clean up is likely not strictly necessary
> to switch to dynamic sizing, but I didn't have a remotely reasonable
> level of confidence in the correctness of the dynamic sizing without first
> doing the clean up.
> 
> Christoffer, I added your Tested-by to the patches that I was confident
> would be fully tested based on the desription of what you tested.  Let me
> know if you disagree with any of 'em.
> 
> v3:
>   - Fix build errors on PPC and MIPS due to missed params during
>     refactoring [kbuild test robot].
>   - Rename the helpers for update_memslots() and add comments describing
>     the new algorithm and how it interacts with searching [Paolo].
>   - Remove the unnecessary and obnoxious warning regarding memslots being
>     a flexible array [Paolo].
>   - Fix typos in the changelog of patch 09/15 [Christoffer].
>   - Collect tags [Christoffer].
> 
> v2:
>   - Split "Drop kvm_arch_create_memslot()" into three patches to move
>     minor functional changes to standalone patches [Janosch].
>   - Rebase to latest kvm/queue (f0574a1cea5b, "KVM: x86: fix ...")
>   - Collect an Acked-by and a Reviewed-by

Paolo, do you want me to rebase this to the latest kvm/queue?
Sean Christopherson Dec. 13, 2019, 8:01 p.m. UTC | #4
On Tue, Dec 03, 2019 at 02:14:33PM -0800, Sean Christopherson wrote:
> On Thu, Oct 24, 2019 at 04:07:29PM -0700, Sean Christopherson wrote:
> > The end goal of this series is to dynamically size the memslot array so
> > that KVM allocates memory based on the number of memslots in use, as
> > opposed to unconditionally allocating memory for the maximum number of
> > memslots.  On x86, each memslot consumes 88 bytes, and so with 2 address
> > spaces of 512 memslots, each VM consumes ~90k bytes for the memslots.
> > E.g. given a VM that uses a total of 30 memslots, dynamic sizing reduces
> > the memory footprint from 90k to ~2.6k bytes.
> > 
> > The changes required to support dynamic sizing are relatively small,
> > e.g. are essentially contained in patches 14/15 and 15/15.  Patches 1-13
> > clean up the memslot code, which has gotten quite crusty, especially
> > __kvm_set_memory_region().  The clean up is likely not strictly necessary
> > to switch to dynamic sizing, but I didn't have a remotely reasonable
> > level of confidence in the correctness of the dynamic sizing without first
> > doing the clean up.
> > 
> > Christoffer, I added your Tested-by to the patches that I was confident
> > would be fully tested based on the desription of what you tested.  Let me
> > know if you disagree with any of 'em.
> > 
> > v3:
> >   - Fix build errors on PPC and MIPS due to missed params during
> >     refactoring [kbuild test robot].
> >   - Rename the helpers for update_memslots() and add comments describing
> >     the new algorithm and how it interacts with searching [Paolo].
> >   - Remove the unnecessary and obnoxious warning regarding memslots being
> >     a flexible array [Paolo].
> >   - Fix typos in the changelog of patch 09/15 [Christoffer].
> >   - Collect tags [Christoffer].
> > 
> > v2:
> >   - Split "Drop kvm_arch_create_memslot()" into three patches to move
> >     minor functional changes to standalone patches [Janosch].
> >   - Rebase to latest kvm/queue (f0574a1cea5b, "KVM: x86: fix ...")
> >   - Collect an Acked-by and a Reviewed-by
> 
> Paolo, do you want me to rebase this to the latest kvm/queue?

Ping.

Applies cleanly on the current kvm/queue and nothing caught fire in
testing (though I only re-tested the series as a whole).
Christian Borntraeger Dec. 16, 2019, 8:25 a.m. UTC | #5
On 13.12.19 21:01, Sean Christopherson wrote:
> On Tue, Dec 03, 2019 at 02:14:33PM -0800, Sean Christopherson wrote:
>> On Thu, Oct 24, 2019 at 04:07:29PM -0700, Sean Christopherson wrote:
>>> The end goal of this series is to dynamically size the memslot array so
>>> that KVM allocates memory based on the number of memslots in use, as
>>> opposed to unconditionally allocating memory for the maximum number of
>>> memslots.  On x86, each memslot consumes 88 bytes, and so with 2 address
>>> spaces of 512 memslots, each VM consumes ~90k bytes for the memslots.
>>> E.g. given a VM that uses a total of 30 memslots, dynamic sizing reduces
>>> the memory footprint from 90k to ~2.6k bytes.
>>>
>>> The changes required to support dynamic sizing are relatively small,
>>> e.g. are essentially contained in patches 14/15 and 15/15.  Patches 1-13
>>> clean up the memslot code, which has gotten quite crusty, especially
>>> __kvm_set_memory_region().  The clean up is likely not strictly necessary
>>> to switch to dynamic sizing, but I didn't have a remotely reasonable
>>> level of confidence in the correctness of the dynamic sizing without first
>>> doing the clean up.
>>>
>>> Christoffer, I added your Tested-by to the patches that I was confident
>>> would be fully tested based on the desription of what you tested.  Let me
>>> know if you disagree with any of 'em.
>>>
>>> v3:
>>>   - Fix build errors on PPC and MIPS due to missed params during
>>>     refactoring [kbuild test robot].
>>>   - Rename the helpers for update_memslots() and add comments describing
>>>     the new algorithm and how it interacts with searching [Paolo].
>>>   - Remove the unnecessary and obnoxious warning regarding memslots being
>>>     a flexible array [Paolo].
>>>   - Fix typos in the changelog of patch 09/15 [Christoffer].
>>>   - Collect tags [Christoffer].
>>>
>>> v2:
>>>   - Split "Drop kvm_arch_create_memslot()" into three patches to move
>>>     minor functional changes to standalone patches [Janosch].
>>>   - Rebase to latest kvm/queue (f0574a1cea5b, "KVM: x86: fix ...")
>>>   - Collect an Acked-by and a Reviewed-by
>>
>> Paolo, do you want me to rebase this to the latest kvm/queue?
> 
> Ping.
> 
> Applies cleanly on the current kvm/queue and nothing caught fire in
> testing (though I only re-tested the series as a whole).

Do you have the latest version somewhere on a branch? The version on the
list no longer applies cleanly.
Sean Christopherson Dec. 17, 2019, 5:55 p.m. UTC | #6
On Mon, Dec 16, 2019 at 09:25:24AM +0100, Christian Borntraeger wrote:
> 
> On 13.12.19 21:01, Sean Christopherson wrote:
> > Applies cleanly on the current kvm/queue and nothing caught fire in
> > testing (though I only re-tested the series as a whole).
> 
> Do you have the latest version somewhere on a branch? The version on the
> list no longer applies cleanly.

Ah, I only tested with my sparse x86-only tree.  The result with three-way
merging, i.e. 'git am -3', looks correct at a glance.

Regardless, I need to spin a new version to handle a conflict with an
unrelated in-flight memslots bug fix, I'll get that sent out today.