mbox series

[0/5] KVM: Turn the vcpu array into an xarray

Message ID 20211105192101.3862492-1-maz@kernel.org (mailing list archive)
Headers show
Series KVM: Turn the vcpu array into an xarray | expand

Message

Marc Zyngier Nov. 5, 2021, 7:20 p.m. UTC
The kvm structure is pretty large. A large portion of it is the vcpu
array, which is 4kB on x86_64 and arm64 as they deal with 512 vcpu
VMs. Of course, hardly anyone runs VMs this big, so this is often a
net waste of memory and cache locality.

A possible approach is to turn the fixed-size array into an xarray,
which results in a net code deletion after a bit of cleanup.

This series is on top of the current linux/master as it touches the
RISC-V implementation. Only tested on arm64.

Marc Zyngier (5):
  KVM: Move wiping of the kvm->vcpus array to common code
  KVM: mips: Use kvm_get_vcpu() instead of open-coded access
  KVM: s390: Use kvm_get_vcpu() instead of open-coded access
  KVM: x86: Use kvm_get_vcpu() instead of open-coded access
  KVM: Convert the kvm->vcpus array to a xarray

 arch/arm64/kvm/arm.c           | 10 +---------
 arch/mips/kvm/loongson_ipi.c   |  4 ++--
 arch/mips/kvm/mips.c           | 23 ++---------------------
 arch/powerpc/kvm/powerpc.c     | 10 +---------
 arch/riscv/kvm/vm.c            | 10 +---------
 arch/s390/kvm/kvm-s390.c       | 26 ++++++--------------------
 arch/x86/kvm/vmx/posted_intr.c |  2 +-
 arch/x86/kvm/x86.c             |  9 +--------
 include/linux/kvm_host.h       |  7 ++++---
 virt/kvm/kvm_main.c            | 33 ++++++++++++++++++++++++++-------
 10 files changed, 45 insertions(+), 89 deletions(-)

Comments

Jürgen Groß Nov. 16, 2021, 2:13 p.m. UTC | #1
On 05.11.21 20:20, Marc Zyngier wrote:
> The kvm structure is pretty large. A large portion of it is the vcpu
> array, which is 4kB on x86_64 and arm64 as they deal with 512 vcpu
> VMs. Of course, hardly anyone runs VMs this big, so this is often a
> net waste of memory and cache locality.
> 
> A possible approach is to turn the fixed-size array into an xarray,
> which results in a net code deletion after a bit of cleanup.
> 
> This series is on top of the current linux/master as it touches the
> RISC-V implementation. Only tested on arm64.
> 
> Marc Zyngier (5):
>    KVM: Move wiping of the kvm->vcpus array to common code
>    KVM: mips: Use kvm_get_vcpu() instead of open-coded access
>    KVM: s390: Use kvm_get_vcpu() instead of open-coded access
>    KVM: x86: Use kvm_get_vcpu() instead of open-coded access
>    KVM: Convert the kvm->vcpus array to a xarray
> 
>   arch/arm64/kvm/arm.c           | 10 +---------
>   arch/mips/kvm/loongson_ipi.c   |  4 ++--
>   arch/mips/kvm/mips.c           | 23 ++---------------------
>   arch/powerpc/kvm/powerpc.c     | 10 +---------
>   arch/riscv/kvm/vm.c            | 10 +---------
>   arch/s390/kvm/kvm-s390.c       | 26 ++++++--------------------
>   arch/x86/kvm/vmx/posted_intr.c |  2 +-
>   arch/x86/kvm/x86.c             |  9 +--------
>   include/linux/kvm_host.h       |  7 ++++---
>   virt/kvm/kvm_main.c            | 33 ++++++++++++++++++++++++++-------
>   10 files changed, 45 insertions(+), 89 deletions(-)
> 

For x86 you can add my:

Tested-by: Juergen Gross <jgross@suse.com>


Juergen
Paolo Bonzini Nov. 16, 2021, 2:21 p.m. UTC | #2
On 11/16/21 15:13, Juergen Gross wrote:
> On 05.11.21 20:20, Marc Zyngier wrote:
>> The kvm structure is pretty large. A large portion of it is the vcpu
>> array, which is 4kB on x86_64 and arm64 as they deal with 512 vcpu
>> VMs. Of course, hardly anyone runs VMs this big, so this is often a
>> net waste of memory and cache locality.
>>
>> A possible approach is to turn the fixed-size array into an xarray,
>> which results in a net code deletion after a bit of cleanup.
>>
>> This series is on top of the current linux/master as it touches the
>> RISC-V implementation. Only tested on arm64.
>>
>> Marc Zyngier (5):
>>    KVM: Move wiping of the kvm->vcpus array to common code
>>    KVM: mips: Use kvm_get_vcpu() instead of open-coded access
>>    KVM: s390: Use kvm_get_vcpu() instead of open-coded access
>>    KVM: x86: Use kvm_get_vcpu() instead of open-coded access
>>    KVM: Convert the kvm->vcpus array to a xarray
>>
>>   arch/arm64/kvm/arm.c           | 10 +---------
>>   arch/mips/kvm/loongson_ipi.c   |  4 ++--
>>   arch/mips/kvm/mips.c           | 23 ++---------------------
>>   arch/powerpc/kvm/powerpc.c     | 10 +---------
>>   arch/riscv/kvm/vm.c            | 10 +---------
>>   arch/s390/kvm/kvm-s390.c       | 26 ++++++--------------------
>>   arch/x86/kvm/vmx/posted_intr.c |  2 +-
>>   arch/x86/kvm/x86.c             |  9 +--------
>>   include/linux/kvm_host.h       |  7 ++++---
>>   virt/kvm/kvm_main.c            | 33 ++++++++++++++++++++++++++-------
>>   10 files changed, 45 insertions(+), 89 deletions(-)
>>
> 
> For x86 you can add my:
> 
> Tested-by: Juergen Gross <jgross@suse.com>

Heh, unfortunately x86 is the only one that needs a change in patch 4. 
I'll Cc you on my version.

Paolo
Jürgen Groß Nov. 16, 2021, 2:54 p.m. UTC | #3
On 16.11.21 15:21, Paolo Bonzini wrote:
> On 11/16/21 15:13, Juergen Gross wrote:
>> On 05.11.21 20:20, Marc Zyngier wrote:
>>> The kvm structure is pretty large. A large portion of it is the vcpu
>>> array, which is 4kB on x86_64 and arm64 as they deal with 512 vcpu
>>> VMs. Of course, hardly anyone runs VMs this big, so this is often a
>>> net waste of memory and cache locality.
>>>
>>> A possible approach is to turn the fixed-size array into an xarray,
>>> which results in a net code deletion after a bit of cleanup.
>>>
>>> This series is on top of the current linux/master as it touches the
>>> RISC-V implementation. Only tested on arm64.
>>>
>>> Marc Zyngier (5):
>>>    KVM: Move wiping of the kvm->vcpus array to common code
>>>    KVM: mips: Use kvm_get_vcpu() instead of open-coded access
>>>    KVM: s390: Use kvm_get_vcpu() instead of open-coded access
>>>    KVM: x86: Use kvm_get_vcpu() instead of open-coded access
>>>    KVM: Convert the kvm->vcpus array to a xarray
>>>
>>>   arch/arm64/kvm/arm.c           | 10 +---------
>>>   arch/mips/kvm/loongson_ipi.c   |  4 ++--
>>>   arch/mips/kvm/mips.c           | 23 ++---------------------
>>>   arch/powerpc/kvm/powerpc.c     | 10 +---------
>>>   arch/riscv/kvm/vm.c            | 10 +---------
>>>   arch/s390/kvm/kvm-s390.c       | 26 ++++++--------------------
>>>   arch/x86/kvm/vmx/posted_intr.c |  2 +-
>>>   arch/x86/kvm/x86.c             |  9 +--------
>>>   include/linux/kvm_host.h       |  7 ++++---
>>>   virt/kvm/kvm_main.c            | 33 ++++++++++++++++++++++++++-------
>>>   10 files changed, 45 insertions(+), 89 deletions(-)
>>>
>>
>> For x86 you can add my:
>>
>> Tested-by: Juergen Gross <jgross@suse.com>
> 
> Heh, unfortunately x86 is the only one that needs a change in patch 4. 
> I'll Cc you on my version.

I guess the changes in kvm_main.c are more important for my series. :-)

I've replaced patch 4 with your variant and everything is still working.
Not sure how relevant that is, though.


Juergen
Paolo Bonzini Nov. 16, 2021, 3:03 p.m. UTC | #4
On 11/5/21 20:20, Marc Zyngier wrote:
> The kvm structure is pretty large. A large portion of it is the vcpu
> array, which is 4kB on x86_64 and arm64 as they deal with 512 vcpu
> VMs. Of course, hardly anyone runs VMs this big, so this is often a
> net waste of memory and cache locality.
> 
> A possible approach is to turn the fixed-size array into an xarray,
> which results in a net code deletion after a bit of cleanup.
> 
> This series is on top of the current linux/master as it touches the
> RISC-V implementation. Only tested on arm64.

Queued, only locally until I get a review for my replacement of patch 4 
(see 
https://lore.kernel.org/kvm/20211116142205.719375-1-pbonzini@redhat.com/T/).

Paolo

> Marc Zyngier (5):
>    KVM: Move wiping of the kvm->vcpus array to common code
>    KVM: mips: Use kvm_get_vcpu() instead of open-coded access
>    KVM: s390: Use kvm_get_vcpu() instead of open-coded access
>    KVM: x86: Use kvm_get_vcpu() instead of open-coded access
>    KVM: Convert the kvm->vcpus array to a xarray
> 
>   arch/arm64/kvm/arm.c           | 10 +---------
>   arch/mips/kvm/loongson_ipi.c   |  4 ++--
>   arch/mips/kvm/mips.c           | 23 ++---------------------
>   arch/powerpc/kvm/powerpc.c     | 10 +---------
>   arch/riscv/kvm/vm.c            | 10 +---------
>   arch/s390/kvm/kvm-s390.c       | 26 ++++++--------------------
>   arch/x86/kvm/vmx/posted_intr.c |  2 +-
>   arch/x86/kvm/x86.c             |  9 +--------
>   include/linux/kvm_host.h       |  7 ++++---
>   virt/kvm/kvm_main.c            | 33 ++++++++++++++++++++++++++-------
>   10 files changed, 45 insertions(+), 89 deletions(-)
>
Marc Zyngier Nov. 16, 2021, 3:40 p.m. UTC | #5
On Tue, 16 Nov 2021 15:03:40 +0000,
Paolo Bonzini <pbonzini@redhat.com> wrote:
> 
> On 11/5/21 20:20, Marc Zyngier wrote:
> > The kvm structure is pretty large. A large portion of it is the vcpu
> > array, which is 4kB on x86_64 and arm64 as they deal with 512 vcpu
> > VMs. Of course, hardly anyone runs VMs this big, so this is often a
> > net waste of memory and cache locality.
> > 
> > A possible approach is to turn the fixed-size array into an xarray,
> > which results in a net code deletion after a bit of cleanup.
> > 
> > This series is on top of the current linux/master as it touches the
> > RISC-V implementation. Only tested on arm64.
> 
> Queued, only locally until I get a review for my replacement of patch
> 4 (see
> https://lore.kernel.org/kvm/20211116142205.719375-1-pbonzini@redhat.com/T/).

In which case, let me send a v2 with the changes that we discussed
with Sean. It will still have my version of patch 4, but that's
nothing you can't fix.

	M.