From patchwork Tue Nov 6 22:53:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Orr X-Patchwork-Id: 10671603 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AF675109C for ; Tue, 6 Nov 2018 22:54:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A16752A6B9 for ; Tue, 6 Nov 2018 22:54:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 954B02A714; Tue, 6 Nov 2018 22:54:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3EDD62A826 for ; Tue, 6 Nov 2018 22:54:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 06A5A6B04A5; Tue, 6 Nov 2018 17:54:08 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 01B0C6B04A7; Tue, 6 Nov 2018 17:54:07 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E4CF96B04A8; Tue, 6 Nov 2018 17:54:07 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ua1-f70.google.com (mail-ua1-f70.google.com [209.85.222.70]) by kanga.kvack.org (Postfix) with ESMTP id B1CE06B04A5 for ; Tue, 6 Nov 2018 17:54:07 -0500 (EST) Received: by mail-ua1-f70.google.com with SMTP id z2so3150137uao.18 for ; Tue, 06 Nov 2018 14:54:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=TOgzYj49o2U7FSAMWCjkV7kVvkSxbFCchh/SWEGRQII=; b=emI8Xu9ig/UlttAWjZ1sSIsxCWfqI9r/aLQphO3aLXOG+NX8qZEAmIl8tceD2meaaa FjMDgMdCHq9cCwr5KgFycPGKCbdbM/nGzQ15kASuX6xf/LiiKsZnuz8Pv7HHPx3U0ATh 91mvEMNv1iiAlmwLE8EqKsM9+2Sj9J+8EiU6yTUAfRpOnkvROdXIkRavhs1ThQ53udie RxYOMgQHZx8N01IGLuW328e9G9clx3PEtIWez/Orp8wEgeB0rbP8oS3r6dfxOyNU7D1X J0/zkZ5wBtsUBtqEBNFjGDXu0YVktGecUVZnYGs5W4wUGLHwdN+36HR32H+WMFU2Rx1V qZtg== X-Gm-Message-State: AGRZ1gIFsNkrGZ+vnGaRJLtyQL+0yX4Crf0/s2QF502RiNn83ZwvQkXi ugPrVWNSTm0Jlh6nNU6/C3iWXBW4526FL9KZBzX+rmD/jnxJRD8a64sVHH3OUgCTUD2iUh0YgHc kbTxrFtELzATsxC7yni+UUgvyfZzk9xpAMeXVNAW2LOH9KpL1xU4TQL654D9LfdoS/jPxWnt/LS iziKagOsoi4YHUr5YblDkU6yZ3g35DoCH10kJ2CZA3zGjvHodeXTKiIOC1j16o/j92hMpAWAodW koCQiF4vB/fVAlkzsKFF0pcezMdVLMZICrjgorrlI5R65l8Tni7rozt70HNQC1GFMz/Gh69zN+G KbVAs5/MN2aChXIzWhP7iUfrjsUsuhKTQC1sXRe9BWLS87ax3fX4DYgPQRHzsr9G+JeSei6iMoD q X-Received: by 2002:ab0:3003:: with SMTP id f3mr13371403ual.80.1541544847337; Tue, 06 Nov 2018 14:54:07 -0800 (PST) X-Received: by 2002:ab0:3003:: with SMTP id f3mr13371391ual.80.1541544846450; Tue, 06 Nov 2018 14:54:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1541544846; cv=none; d=google.com; s=arc-20160816; b=P0y4M9LDxaXKzyavFqPi9wXTZpM3KkyO91u8nudBasOaLro2WnnW1GYlaF2fZB6pa0 hj7aLxaGGpYkydVKO7MH13UukNOznjNnfUXbNlt5UT6zyDpWwhRrMsEMbPkbR0eFPkqU VsGVC0buJnnMypgiqcJkjKpjv+lzDV193n7WQQ6zvMxaln0RED0VsWQz3PyJaHMtey3T A0OPY7Pjeo9OANmUIWx2LXLbntDagn0QRu73KCQ2BAfAmc8/ydQ8BID6XXYMbNWEPgSe E6bX5a145FpkzL8mHNpnXoQAB788tjT/0vGIuBD8bP7lkWVgvh3Zga3lnVzSBogvmoNt zwzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:dkim-signature; bh=TOgzYj49o2U7FSAMWCjkV7kVvkSxbFCchh/SWEGRQII=; b=R/CzcivwWnnC+3D2s8NLAntww7RQLKZlDN1pF85jostBgyu4RPHqN5Z6qtCnofuBkn y+S7n8dXocI3VvevTu6+ctBH8bzBn14jY5Xh4Qvr+m+92T7cM555KB3mq9zcYLqoNNC/ Ef/BqwsGVTRvd3a9qSKyTDfBse8v47ygMBEk9ODOyXNW1fwjy25gqd7hhaMWlto7xABp 7UxqT6u05cIk3Lnz3Ipf0CkYSol3DLkQjdzPB75TGMYcdoZCnI8H+jo4YC4tr1dxdzEf uHpBwFTM9l6N24zZVkXAnFIpawKyV7PEOvvmq/qr5QI0M7gBib6EWpamr4VUAN/2/GgB ossA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=rXsnHZMv; spf=pass (google.com: domain of 3jhviwwckcdmbpgrdggvddvat.rdbaxcjm-bbzkprz.dgv@flex--marcorr.bounces.google.com designates 209.85.220.73 as permitted sender) smtp.mailfrom=3jhviWwcKCDMbPgRdggVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--marcorr.bounces.google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f73.google.com (mail-sor-f73.google.com. [209.85.220.73]) by mx.google.com with SMTPS id y134sor17067492vkd.70.2018.11.06.14.54.06 for (Google Transport Security); Tue, 06 Nov 2018 14:54:06 -0800 (PST) Received-SPF: pass (google.com: domain of 3jhviwwckcdmbpgrdggvddvat.rdbaxcjm-bbzkprz.dgv@flex--marcorr.bounces.google.com designates 209.85.220.73 as permitted sender) client-ip=209.85.220.73; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=rXsnHZMv; spf=pass (google.com: domain of 3jhviwwckcdmbpgrdggvddvat.rdbaxcjm-bbzkprz.dgv@flex--marcorr.bounces.google.com designates 209.85.220.73 as permitted sender) smtp.mailfrom=3jhviWwcKCDMbPgRdggVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--marcorr.bounces.google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=TOgzYj49o2U7FSAMWCjkV7kVvkSxbFCchh/SWEGRQII=; b=rXsnHZMvf8wTrqocYZLf71izwF3oxXYBsyWtaVha5PVlRSTPBJl1gevpEGnYSI4Ub/ ufPGB6ZCQOmas4iGE0qVy4LI0FN8G7PD9ZavIkEm4BsUi6DIXMt2gTGW87r4R6kJedGH Kh/xoXUq3txSabwYLpHoAksLYpEB6/EmAb9HtooAhfetWg1ICTNPFmHVrLm+5+ekn2AI SFHdt36iwRN21pmohfn6lPASb9aYITc6dgIj3o/Fr96TshoQeT64W0pE0KDaoNsUMCsp LkCDd5WOpPHfynJpyshMQ/8FRmatBzzKrjChHmXo1EZtD0TC9UP4QM9DgYvEGk3c3zPK hHkg== X-Google-Smtp-Source: AJdET5dqGd7aNP+ew05L/W/7tOOB+A6U4CNobf+WkjgyQAq+o4IjKxulE+py9fpMPzZKKAKoce9PxfxgWPJr X-Received: by 2002:a1f:97cb:: with SMTP id z194mr15228315vkd.2.1541544846145; Tue, 06 Nov 2018 14:54:06 -0800 (PST) Date: Tue, 6 Nov 2018 14:53:56 -0800 In-Reply-To: <20181106225356.119901-1-marcorr@google.com> Message-Id: <20181106225356.119901-3-marcorr@google.com> Mime-Version: 1.0 References: <20181106225356.119901-1-marcorr@google.com> X-Mailer: git-send-email 2.19.1.930.g4563a0d9d0-goog Subject: [kvm PATCH v8 2/2] kvm: x86: Dynamically allocate guest_fpu From: Marc Orr To: kvm@vger.kernel.org, jmattson@google.com, rientjes@google.com, konrad.wilk@oracle.com, linux-mm@kvack.org, akpm@linux-foundation.org, pbonzini@redhat.com, rkrcmar@redhat.com, willy@infradead.org, sean.j.christopherson@intel.com, dave.hansen@linux.intel.com, kernellwp@gmail.com Cc: Marc Orr , Dave Hansen X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Previously, the guest_fpu field was embedded in the kvm_vcpu_arch struct. Unfortunately, the field is quite large, (e.g., 4352 bytes on my current setup). This bloats the kvm_vcpu_arch struct for x86 into an order 3 memory allocation, which can become a problem on overcommitted machines. Thus, this patch moves the fpu state outside of the kvm_vcpu_arch struct. With this patch applied, the kvm_vcpu_arch struct is reduced to 15168 bytes for vmx on my setup when building the kernel with kvmconfig. Suggested-by: Dave Hansen Signed-off-by: Marc Orr --- arch/x86/include/asm/kvm_host.h | 3 +- arch/x86/kvm/svm.c | 10 +++++++ arch/x86/kvm/vmx.c | 10 +++++++ arch/x86/kvm/x86.c | 51 ++++++++++++++++++++++++--------- 4 files changed, 60 insertions(+), 14 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index ebb1d7a755d4..c8a2a263f91f 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -610,7 +610,7 @@ struct kvm_vcpu_arch { * "guest_fpu" state here contains the guest FPU context, with the * host PRKU bits. */ - struct fpu guest_fpu; + struct fpu *guest_fpu; u64 xcr0; u64 guest_supported_xcr0; @@ -1194,6 +1194,7 @@ struct kvm_arch_async_pf { }; extern struct kvm_x86_ops *kvm_x86_ops; +extern struct kmem_cache *x86_fpu_cache; #define __KVM_HAVE_ARCH_VM_ALLOC static inline struct kvm *kvm_arch_alloc_vm(void) diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index f416f5c7f2ae..ac0c52ca22c6 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -2121,6 +2121,13 @@ static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id) goto out; } + svm->vcpu.arch.guest_fpu = kmem_cache_zalloc(x86_fpu_cache, GFP_KERNEL); + if (!svm->vcpu.arch.guest_fpu) { + printk(KERN_ERR "kvm: failed to allocate vcpu's fpu\n"); + err = -ENOMEM; + goto free_partial_svm; + } + err = kvm_vcpu_init(&svm->vcpu, kvm, id); if (err) goto free_svm; @@ -2180,6 +2187,8 @@ static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id) uninit: kvm_vcpu_uninit(&svm->vcpu); free_svm: + kmem_cache_free(x86_fpu_cache, svm->vcpu.arch.guest_fpu); +free_partial_svm: kmem_cache_free(kvm_vcpu_cache, svm); out: return ERR_PTR(err); @@ -2194,6 +2203,7 @@ static void svm_free_vcpu(struct kvm_vcpu *vcpu) __free_page(virt_to_page(svm->nested.hsave)); __free_pages(virt_to_page(svm->nested.msrpm), MSRPM_ALLOC_ORDER); kvm_vcpu_uninit(vcpu); + kmem_cache_free(x86_fpu_cache, svm->vcpu.arch.guest_fpu); kmem_cache_free(kvm_vcpu_cache, svm); /* * The vmcb page can be recycled, causing a false negative in diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index abeeb45d1c33..4078cf15a4b0 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -11476,6 +11476,7 @@ static void vmx_free_vcpu(struct kvm_vcpu *vcpu) free_loaded_vmcs(vmx->loaded_vmcs); kfree(vmx->guest_msrs); kvm_vcpu_uninit(vcpu); + kmem_cache_free(x86_fpu_cache, vmx->vcpu.arch.guest_fpu); kmem_cache_free(kvm_vcpu_cache, vmx); } @@ -11489,6 +11490,13 @@ static struct kvm_vcpu *vmx_create_vcpu(struct kvm *kvm, unsigned int id) if (!vmx) return ERR_PTR(-ENOMEM); + vmx->vcpu.arch.guest_fpu = kmem_cache_zalloc(x86_fpu_cache, GFP_KERNEL); + if (!vmx->vcpu.arch.guest_fpu) { + printk(KERN_ERR "kvm: failed to allocate vcpu's fpu\n"); + err = -ENOMEM; + goto free_partial_vcpu; + } + vmx->vpid = allocate_vpid(); err = kvm_vcpu_init(&vmx->vcpu, kvm, id); @@ -11576,6 +11584,8 @@ static struct kvm_vcpu *vmx_create_vcpu(struct kvm *kvm, unsigned int id) kvm_vcpu_uninit(&vmx->vcpu); free_vcpu: free_vpid(vmx->vpid); + kmem_cache_free(x86_fpu_cache, vmx->vcpu.arch.guest_fpu); +free_partial_vcpu: kmem_cache_free(kvm_vcpu_cache, vmx); return ERR_PTR(err); } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ff77514f7367..d81ca4e7ae57 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -213,6 +213,9 @@ struct kvm_stats_debugfs_item debugfs_entries[] = { u64 __read_mostly host_xcr0; +struct kmem_cache *x86_fpu_cache; +EXPORT_SYMBOL_GPL(x86_fpu_cache); + static int emulator_fix_hypercall(struct x86_emulate_ctxt *ctxt); static inline void kvm_async_pf_hash_reset(struct kvm_vcpu *vcpu) @@ -3635,7 +3638,7 @@ static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu, static void fill_xsave(u8 *dest, struct kvm_vcpu *vcpu) { - struct xregs_state *xsave = &vcpu->arch.guest_fpu.state.xsave; + struct xregs_state *xsave = &vcpu->arch.guest_fpu->state.xsave; u64 xstate_bv = xsave->header.xfeatures; u64 valid; @@ -3677,7 +3680,7 @@ static void fill_xsave(u8 *dest, struct kvm_vcpu *vcpu) static void load_xsave(struct kvm_vcpu *vcpu, u8 *src) { - struct xregs_state *xsave = &vcpu->arch.guest_fpu.state.xsave; + struct xregs_state *xsave = &vcpu->arch.guest_fpu->state.xsave; u64 xstate_bv = *(u64 *)(src + XSAVE_HDR_OFFSET); u64 valid; @@ -3725,7 +3728,7 @@ static void kvm_vcpu_ioctl_x86_get_xsave(struct kvm_vcpu *vcpu, fill_xsave((u8 *) guest_xsave->region, vcpu); } else { memcpy(guest_xsave->region, - &vcpu->arch.guest_fpu.state.fxsave, + &vcpu->arch.guest_fpu->state.fxsave, sizeof(struct fxregs_state)); *(u64 *)&guest_xsave->region[XSAVE_HDR_OFFSET / sizeof(u32)] = XFEATURE_MASK_FPSSE; @@ -3755,7 +3758,7 @@ static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu, if (xstate_bv & ~XFEATURE_MASK_FPSSE || mxcsr & ~mxcsr_feature_mask) return -EINVAL; - memcpy(&vcpu->arch.guest_fpu.state.fxsave, + memcpy(&vcpu->arch.guest_fpu->state.fxsave, guest_xsave->region, sizeof(struct fxregs_state)); } return 0; @@ -6818,11 +6821,30 @@ int kvm_arch_init(void *opaque) goto out; } + /* + * KVM explicitly assumes that the guest has an FPU and + * FXSAVE/FXRSTOR. For example, the KVM_GET_FPU explicitly casts the + * vCPU's FPU state as a fxregs_state struct. + */ + if (!boot_cpu_has(X86_FEATURE_FPU) || !boot_cpu_has(X86_FEATURE_FXSR)) { + printk(KERN_ERR "kvm: inadequate fpu\n"); + r = -EOPNOTSUPP; + goto out; + } + r = -ENOMEM; + x86_fpu_cache = kmem_cache_create("x86_fpu", fpu_kernel_xstate_size, + __alignof__(struct fpu), SLAB_ACCOUNT, + NULL); + if (!x86_fpu_cache) { + printk(KERN_ERR "kvm: failed to allocate cache for x86 fpu\n"); + goto out; + } + shared_msrs = alloc_percpu(struct kvm_shared_msrs); if (!shared_msrs) { printk(KERN_ERR "kvm: failed to allocate percpu kvm_shared_msrs\n"); - goto out; + goto out_free_x86_fpu_cache; } r = kvm_mmu_module_init(); @@ -6855,6 +6877,8 @@ int kvm_arch_init(void *opaque) out_free_percpu: free_percpu(shared_msrs); +out_free_x86_fpu_cache: + kmem_cache_destroy(x86_fpu_cache); out: return r; } @@ -6878,6 +6902,7 @@ void kvm_arch_exit(void) kvm_x86_ops = NULL; kvm_mmu_module_exit(); free_percpu(shared_msrs); + kmem_cache_destroy(x86_fpu_cache); } int kvm_vcpu_halt(struct kvm_vcpu *vcpu) @@ -8001,7 +8026,7 @@ static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu) preempt_disable(); copy_fpregs_to_fpstate(¤t->thread.fpu); /* PKRU is separately restored in kvm_x86_ops->run. */ - __copy_kernel_to_fpregs(&vcpu->arch.guest_fpu.state, + __copy_kernel_to_fpregs(&vcpu->arch.guest_fpu->state, ~XFEATURE_MASK_PKRU); preempt_enable(); trace_kvm_fpu(1); @@ -8011,7 +8036,7 @@ static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu) static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu) { preempt_disable(); - copy_fpregs_to_fpstate(&vcpu->arch.guest_fpu); + copy_fpregs_to_fpstate(vcpu->arch.guest_fpu); copy_kernel_to_fpregs(¤t->thread.fpu.state); preempt_enable(); ++vcpu->stat.fpu_reload; @@ -8506,7 +8531,7 @@ int kvm_arch_vcpu_ioctl_get_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu) vcpu_load(vcpu); - fxsave = &vcpu->arch.guest_fpu.state.fxsave; + fxsave = &vcpu->arch.guest_fpu->state.fxsave; memcpy(fpu->fpr, fxsave->st_space, 128); fpu->fcw = fxsave->cwd; fpu->fsw = fxsave->swd; @@ -8526,7 +8551,7 @@ int kvm_arch_vcpu_ioctl_set_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu) vcpu_load(vcpu); - fxsave = &vcpu->arch.guest_fpu.state.fxsave; + fxsave = &vcpu->arch.guest_fpu->state.fxsave; memcpy(fxsave->st_space, fpu->fpr, 128); fxsave->cwd = fpu->fcw; @@ -8582,9 +8607,9 @@ static int sync_regs(struct kvm_vcpu *vcpu) static void fx_init(struct kvm_vcpu *vcpu) { - fpstate_init(&vcpu->arch.guest_fpu.state); + fpstate_init(&vcpu->arch.guest_fpu->state); if (boot_cpu_has(X86_FEATURE_XSAVES)) - vcpu->arch.guest_fpu.state.xsave.header.xcomp_bv = + vcpu->arch.guest_fpu->state.xsave.header.xcomp_bv = host_xcr0 | XSTATE_COMPACTION_ENABLED; /* @@ -8708,11 +8733,11 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) */ if (init_event) kvm_put_guest_fpu(vcpu); - mpx_state_buffer = get_xsave_addr(&vcpu->arch.guest_fpu.state.xsave, + mpx_state_buffer = get_xsave_addr(&vcpu->arch.guest_fpu->state.xsave, XFEATURE_MASK_BNDREGS); if (mpx_state_buffer) memset(mpx_state_buffer, 0, sizeof(struct mpx_bndreg_state)); - mpx_state_buffer = get_xsave_addr(&vcpu->arch.guest_fpu.state.xsave, + mpx_state_buffer = get_xsave_addr(&vcpu->arch.guest_fpu->state.xsave, XFEATURE_MASK_BNDCSR); if (mpx_state_buffer) memset(mpx_state_buffer, 0, sizeof(struct mpx_bndcsr));