From patchwork Fri Sep 20 21:24:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154887 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A89B615E6 for ; Fri, 20 Sep 2019 21:26:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 90E762086A for ; Fri, 20 Sep 2019 21:26:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387431AbfITVZM (ORCPT ); Fri, 20 Sep 2019 17:25:12 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57976 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729784AbfITVZM (ORCPT ); Fri, 20 Sep 2019 17:25:12 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 80B048980F5; Fri, 20 Sep 2019 21:25:12 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0833B10013D9; Fri, 20 Sep 2019 21:25:10 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 01/17] x86: spec_ctrl: fix SPEC_CTRL initialization after kexec Date: Fri, 20 Sep 2019 17:24:53 -0400 Message-Id: <20190920212509.2578-2-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.67]); Fri, 20 Sep 2019 21:25:12 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org We can't assume the SPEC_CTRL msr is zero at boot because it could be left enabled by a previous kernel booted with spec_store_bypass_disable=on. Without this fix a boot with spec_store_bypass_disable=on followed by a kexec boot with spec_store_bypass_disable=off would erroneously and unexpectedly leave bit 2 set in SPEC_CTRL. Signed-off-by: Andrea Arcangeli --- arch/x86/include/asm/msr-index.h | 2 ++ arch/x86/kernel/cpu/bugs.c | 20 +++++++++++++++++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 20ce682a2540..3ba95728a6fe 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -47,6 +47,8 @@ #define SPEC_CTRL_STIBP BIT(SPEC_CTRL_STIBP_SHIFT) /* STIBP mask */ #define SPEC_CTRL_SSBD_SHIFT 2 /* Speculative Store Bypass Disable bit */ #define SPEC_CTRL_SSBD BIT(SPEC_CTRL_SSBD_SHIFT) /* Speculative Store Bypass Disable */ +#define SPEC_CTRL_ALL (SPEC_CTRL_IBRS|SPEC_CTRL_STIBP| \ + SPEC_CTRL_SSBD) /* all known bits */ #define MSR_IA32_PRED_CMD 0x00000049 /* Prediction Command */ #define PRED_CMD_IBPB BIT(0) /* Indirect Branch Prediction Barrier */ diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index 91c2561b905f..e3922dcf252f 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -92,8 +92,26 @@ void __init check_bugs(void) * have unknown values. AMD64_LS_CFG MSR is cached in the early AMD * init code as it is not enumerated and depends on the family. */ - if (boot_cpu_has(X86_FEATURE_MSR_SPEC_CTRL)) + if (boot_cpu_has(X86_FEATURE_MSR_SPEC_CTRL)) { rdmsrl(MSR_IA32_SPEC_CTRL, x86_spec_ctrl_base); + /* + * Clear the non reserved bits from x86_spec_ctrl_base + * to fix kexec. Otherwise for example SSBD could be + * left enabled despite booting with + * spec_store_bypass_disable=off because SSBD would be + * erroneously mistaken as a reserved bit set by the + * BIOS when in fact it was set by a previous kernel + * booted with spec_store_bypass_disable=on. Careful + * however not to write SPEC_CTRL unnecessarily to + * keep the virt MSR intercept enabled as long as + * possible. + */ + if (x86_spec_ctrl_base & SPEC_CTRL_ALL) { + /* all known bits must not be set at boot, clear it */ + x86_spec_ctrl_base &= ~SPEC_CTRL_ALL; + wrmsrl(MSR_IA32_SPEC_CTRL, x86_spec_ctrl_base); + } + } /* Allow STIBP in MSR_SPEC_CTRL if supported */ if (boot_cpu_has(X86_FEATURE_STIBP)) From patchwork Fri Sep 20 21:24:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154869 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8FE4F15E6 for ; Fri, 20 Sep 2019 21:25:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4F9332173E for ; Fri, 20 Sep 2019 21:25:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393782AbfITVZO (ORCPT ); Fri, 20 Sep 2019 17:25:14 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54162 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730150AbfITVZN (ORCPT ); Fri, 20 Sep 2019 17:25:13 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 876A03082E24; Fri, 20 Sep 2019 21:25:12 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F175C5D6B2; Fri, 20 Sep 2019 21:25:09 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 02/17] KVM: monolithic: x86: convert the kvm_x86_ops methods to external functions Date: Fri, 20 Sep 2019 17:24:54 -0400 Message-Id: <20190920212509.2578-3-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Fri, 20 Sep 2019 21:25:12 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This replaces all kvm_x86_ops pointer to functions with regular external functions that don't require indirect calls. This is the first commit of a series that aim to replace the modular kvm.ko kernel module with a monolithic kvm-intel/kvm-amd model. This change has the only possible cons of wasting some disk space in /lib/modules/. The pros are that it saves CPUS and some minor RAM which are more scarse resources than disk space. The pointer to function virtual template model cannot provide any runtime benefit because kvm-intel and kvm-amd can't be loaded at the same time. In practice this optimization results in a double digit percent reduction in the vmexit latency with the default retpoline spectre v2 mitigation enabled in the host. When the host is booted with spectre_v2=off this still results in a measurable improvement of the order of 1% depending on the architecture and workload. Supposedly userland workloads in guest making lots of use of the BTB will benefit more. To reduce the rejecting parts while tracking upstream, this doesn't attempt to entirely remove the kvm_x86_ops structure yet, that is meant for a later cleanup. The pmu ops have been already cleaned up in this patchset because it was left completely unused right after the conversion from pointer to functions to external functions. Further incremental minor optimizations that weren't possible before are now enabled by the monolithic model. For example it would be possible later to convert some of the small external methods to inline functions from the {svm,vmx}_ops.c file to kvm_ops.h. However that will require more Makefile tweaks. This is a list of the most common retpolines executed in KVM on VMX under a guest workload triggering a high resolution timer SIGALRM flood before the monolithic KVM patchset is applied. [..] @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 cancel_hv_timer.isra.46+44 restart_apic_timer+295 kvm_set_msr_common+1435 vmx_set_msr+478 handle_wrmsr+85 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 65382 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+1646 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 66164 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 kvm_read_l1_tsc+41 __kvm_wait_lapic_expire+60 vmx_vcpu_run.part.88+1091 vcpu_enter_guest+423 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 66199 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+4958 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 66227 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 restart_apic_timer+99 kvm_set_msr_common+1435 vmx_set_msr+478 handle_wrmsr+85 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 130619 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 kvm_read_l1_tsc+41 vmx_set_hv_timer+81 restart_apic_timer+99 kvm_set_msr_common+1435 vmx_set_msr+478 handle_wrmsr+85 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 130665 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 kvm_skip_emulated_instruction+49 handle_wrmsr+102 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 131020 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 kvm_skip_emulated_instruction+82 handle_wrmsr+102 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 131025 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 handle_wrmsr+85 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 131043 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 skip_emulated_instruction+48 kvm_skip_emulated_instruction+82 handle_wrmsr+102 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 131046 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+4009 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 132405 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rcx+33 vcpu_enter_guest+1689 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 197697 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vmx_vcpu_run.part.88+358 vcpu_enter_guest+423 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 198736 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+575 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 198771 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+423 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 198793 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+486 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 198801 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+168 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 198848 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 397680 @total: 3816655 Here the same but on SVM: [..] @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 clockevents_program_event+148 hrtimer_start_range_ns+528 start_sw_timer+356 restart_apic_timer+111 kvm_set_msr_common+1435 msr_interception+138 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 36031 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 lapic_next_event+28 clockevents_program_event+148 hrtimer_start_range_ns+528 start_sw_timer+356 restart_apic_timer+111 kvm_set_msr_common+1435 msr_interception+138 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 36063 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 ktime_get+58 clockevents_program_event+84 hrtimer_try_to_cancel+168 hrtimer_cancel+21 kvm_set_lapic_tscdeadline_msr+43 kvm_set_msr_common+1435 msr_interception+138 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 36134 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 lapic_next_event+28 clockevents_program_event+148 hrtimer_try_to_cancel+168 hrtimer_cancel+21 kvm_set_lapic_tscdeadline_msr+43 kvm_set_msr_common+1435 msr_interception+138 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 36146 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 clockevents_program_event+148 hrtimer_try_to_cancel+168 hrtimer_cancel+21 kvm_set_lapic_tscdeadline_msr+43 kvm_set_msr_common+1435 msr_interception+138 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 36190 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 ktime_get+58 clockevents_program_event+84 hrtimer_start_range_ns+528 start_sw_timer+356 restart_apic_timer+111 kvm_set_msr_common+1435 msr_interception+138 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 36281 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+1646 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 37752 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 kvm_read_l1_tsc+41 __kvm_wait_lapic_expire+60 svm_vcpu_run+1276 ]: 37886 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+4958 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 37957 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 kvm_read_l1_tsc+41 start_sw_timer+302 restart_apic_timer+111 kvm_set_msr_common+1435 msr_interception+138 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 74358 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 ktime_get+58 start_sw_timer+279 restart_apic_timer+111 kvm_set_msr_common+1435 msr_interception+138 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 74558 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 kvm_skip_emulated_instruction+82 msr_interception+356 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 74713 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 kvm_skip_emulated_instruction+49 msr_interception+356 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 74757 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 msr_interception+138 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 74795 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 kvm_get_rflags+28 svm_interrupt_allowed+50 vcpu_enter_guest+4009 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 75647 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+4009 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 75812 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rcx+33 vcpu_enter_guest+1689 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 112579 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+575 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 113371 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+423 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 113386 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+486 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 113414 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+168 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 113601 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 vcpu_enter_guest+772 kvm_arch_vcpu_ioctl_run+263 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 227076 @total: 3829460 Signed-off-by: Andrea Arcangeli --- arch/x86/include/asm/kvm_ops.h | 166 ++++++++ arch/x86/kvm/svm_ops.c | 672 +++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx_ops.c | 672 +++++++++++++++++++++++++++++++++ 3 files changed, 1510 insertions(+) create mode 100644 arch/x86/include/asm/kvm_ops.h create mode 100644 arch/x86/kvm/svm_ops.c create mode 100644 arch/x86/kvm/vmx/vmx_ops.c diff --git a/arch/x86/include/asm/kvm_ops.h b/arch/x86/include/asm/kvm_ops.h new file mode 100644 index 000000000000..1ec5380c9b67 --- /dev/null +++ b/arch/x86/include/asm/kvm_ops.h @@ -0,0 +1,166 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _ASM_X86_KVM_OPS_H +#define _ASM_X86_KVM_OPS_H + +extern __init int kvm_x86_ops_cpu_has_kvm_support(void); +extern __init int kvm_x86_ops_disabled_by_bios(void); +extern int kvm_x86_ops_hardware_enable(void); +extern void kvm_x86_ops_hardware_disable(void); +extern __init int kvm_x86_ops_check_processor_compatibility(void); +extern __init int kvm_x86_ops_hardware_setup(void); +extern void kvm_x86_ops_hardware_unsetup(void); +extern bool kvm_x86_ops_cpu_has_accelerated_tpr(void); +extern bool kvm_x86_ops_has_emulated_msr(int index); +extern void kvm_x86_ops_cpuid_update(struct kvm_vcpu *vcpu); +extern struct kvm *kvm_x86_ops_vm_alloc(void); +extern void kvm_x86_ops_vm_free(struct kvm *kvm); +extern int kvm_x86_ops_vm_init(struct kvm *kvm); +extern void kvm_x86_ops_vm_destroy(struct kvm *kvm); +extern struct kvm_vcpu *kvm_x86_ops_vcpu_create(struct kvm *kvm, unsigned id); +extern void kvm_x86_ops_vcpu_free(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event); +extern void kvm_x86_ops_prepare_guest_switch(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_vcpu_load(struct kvm_vcpu *vcpu, int cpu); +extern void kvm_x86_ops_vcpu_put(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_update_bp_intercept(struct kvm_vcpu *vcpu); +extern int kvm_x86_ops_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr); +extern int kvm_x86_ops_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr); +extern u64 kvm_x86_ops_get_segment_base(struct kvm_vcpu *vcpu, int seg); +extern void kvm_x86_ops_get_segment(struct kvm_vcpu *vcpu, + struct kvm_segment *var, int seg); +extern int kvm_x86_ops_get_cpl(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_set_segment(struct kvm_vcpu *vcpu, + struct kvm_segment *var, int seg); +extern void kvm_x86_ops_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, + int *l); +extern void kvm_x86_ops_decache_cr0_guest_bits(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_decache_cr3(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_decache_cr4_guest_bits(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0); +extern void kvm_x86_ops_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3); +extern int kvm_x86_ops_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4); +extern void kvm_x86_ops_set_efer(struct kvm_vcpu *vcpu, u64 efer); +extern void kvm_x86_ops_get_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt); +extern void kvm_x86_ops_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt); +extern void kvm_x86_ops_get_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt); +extern void kvm_x86_ops_set_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt); +extern u64 kvm_x86_ops_get_dr6(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_set_dr6(struct kvm_vcpu *vcpu, unsigned long value); +extern void kvm_x86_ops_sync_dirty_debug_regs(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_set_dr7(struct kvm_vcpu *vcpu, unsigned long value); +extern void kvm_x86_ops_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg); +extern unsigned long kvm_x86_ops_get_rflags(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_set_rflags(struct kvm_vcpu *vcpu, + unsigned long rflags); +extern void kvm_x86_ops_tlb_flush(struct kvm_vcpu *vcpu, bool invalidate_gpa); +extern int kvm_x86_ops_tlb_remote_flush(struct kvm *kvm); +extern int kvm_x86_ops_tlb_remote_flush_with_range(struct kvm *kvm, + struct kvm_tlb_range *range); +extern void kvm_x86_ops_tlb_flush_gva(struct kvm_vcpu *vcpu, gva_t addr); +extern void kvm_x86_ops_run(struct kvm_vcpu *vcpu); +extern int kvm_x86_ops_handle_exit(struct kvm_vcpu *vcpu); +extern int kvm_x86_ops_skip_emulated_instruction(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask); +extern u32 kvm_x86_ops_get_interrupt_shadow(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_patch_hypercall(struct kvm_vcpu *vcpu, + unsigned char *hypercall_addr); +extern void kvm_x86_ops_set_irq(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_set_nmi(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_queue_exception(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_cancel_injection(struct kvm_vcpu *vcpu); +extern int kvm_x86_ops_interrupt_allowed(struct kvm_vcpu *vcpu); +extern int kvm_x86_ops_nmi_allowed(struct kvm_vcpu *vcpu); +extern bool kvm_x86_ops_get_nmi_mask(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked); +extern void kvm_x86_ops_enable_nmi_window(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_enable_irq_window(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, + int irr); +extern bool kvm_x86_ops_get_enable_apicv(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr); +extern void kvm_x86_ops_hwapic_isr_update(struct kvm_vcpu *vcpu, int isr); +extern bool kvm_x86_ops_guest_apic_has_interrupt(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_load_eoi_exitmap(struct kvm_vcpu *vcpu, + u64 *eoi_exit_bitmap); +extern void kvm_x86_ops_set_virtual_apic_mode(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_set_apic_access_page_addr(struct kvm_vcpu *vcpu, + hpa_t hpa); +extern void kvm_x86_ops_deliver_posted_interrupt(struct kvm_vcpu *vcpu, + int vector); +extern int kvm_x86_ops_sync_pir_to_irr(struct kvm_vcpu *vcpu); +extern int kvm_x86_ops_set_tss_addr(struct kvm *kvm, unsigned int addr); +extern int kvm_x86_ops_set_identity_map_addr(struct kvm *kvm, u64 ident_addr); +extern int kvm_x86_ops_get_tdp_level(struct kvm_vcpu *vcpu); +extern u64 kvm_x86_ops_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, + bool is_mmio); +extern int kvm_x86_ops_get_lpage_level(void); +extern bool kvm_x86_ops_rdtscp_supported(void); +extern bool kvm_x86_ops_invpcid_supported(void); +extern void kvm_x86_ops_set_tdp_cr3(struct kvm_vcpu *vcpu, unsigned long cr3); +extern void kvm_x86_ops_set_supported_cpuid(u32 func, + struct kvm_cpuid_entry2 *entry); +extern bool kvm_x86_ops_has_wbinvd_exit(void); +extern u64 kvm_x86_ops_read_l1_tsc_offset(struct kvm_vcpu *vcpu); +extern u64 kvm_x86_ops_write_l1_tsc_offset(struct kvm_vcpu *vcpu, u64 offset); +extern void kvm_x86_ops_get_exit_info(struct kvm_vcpu *vcpu, u64 *info1, + u64 *info2); +extern int kvm_x86_ops_check_intercept(struct kvm_vcpu *vcpu, + struct x86_instruction_info *info, + enum x86_intercept_stage stage); +extern void kvm_x86_ops_handle_exit_irqoff(struct kvm_vcpu *vcpu); +extern bool kvm_x86_ops_mpx_supported(void); +extern bool kvm_x86_ops_xsaves_supported(void); +extern bool kvm_x86_ops_umip_emulated(void); +extern bool kvm_x86_ops_pt_supported(void); +extern int kvm_x86_ops_check_nested_events(struct kvm_vcpu *vcpu, + bool external_intr); +extern void kvm_x86_ops_request_immediate_exit(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_sched_in(struct kvm_vcpu *kvm, int cpu); +extern void kvm_x86_ops_slot_enable_log_dirty(struct kvm *kvm, + struct kvm_memory_slot *slot); +extern void kvm_x86_ops_slot_disable_log_dirty(struct kvm *kvm, + struct kvm_memory_slot *slot); +extern void kvm_x86_ops_flush_log_dirty(struct kvm *kvm); +extern void kvm_x86_ops_enable_log_dirty_pt_masked(struct kvm *kvm, + struct kvm_memory_slot *slot, + gfn_t offset, + unsigned long mask); +extern int kvm_x86_ops_write_log_dirty(struct kvm_vcpu *vcpu); +extern int kvm_x86_ops_pre_block(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_post_block(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_vcpu_blocking(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_vcpu_unblocking(struct kvm_vcpu *vcpu); +extern int kvm_x86_ops_update_pi_irte(struct kvm *kvm, unsigned int host_irq, + uint32_t guest_irq, bool set); +extern void kvm_x86_ops_apicv_post_state_restore(struct kvm_vcpu *vcpu); +extern bool kvm_x86_ops_dy_apicv_has_pending_interrupt(struct kvm_vcpu *vcpu); +extern int kvm_x86_ops_set_hv_timer(struct kvm_vcpu *vcpu, + u64 guest_deadline_tsc, bool *expired); +extern void kvm_x86_ops_cancel_hv_timer(struct kvm_vcpu *vcpu); +extern void kvm_x86_ops_setup_mce(struct kvm_vcpu *vcpu); +extern int kvm_x86_ops_get_nested_state(struct kvm_vcpu *vcpu, + struct kvm_nested_state __user *user_kvm_nested_state, + unsigned user_data_size); +extern int kvm_x86_ops_set_nested_state(struct kvm_vcpu *vcpu, + struct kvm_nested_state __user *user_kvm_nested_state, + struct kvm_nested_state *kvm_state); +extern void kvm_x86_ops_get_vmcs12_pages(struct kvm_vcpu *vcpu); +extern int kvm_x86_ops_smi_allowed(struct kvm_vcpu *vcpu); +extern int kvm_x86_ops_pre_enter_smm(struct kvm_vcpu *vcpu, char *smstate); +extern int kvm_x86_ops_pre_leave_smm(struct kvm_vcpu *vcpu, + const char *smstate); +extern int kvm_x86_ops_enable_smi_window(struct kvm_vcpu *vcpu); +extern int kvm_x86_ops_mem_enc_op(struct kvm *kvm, void __user *argp); +extern int kvm_x86_ops_mem_enc_reg_region(struct kvm *kvm, + struct kvm_enc_region *argp); +extern int kvm_x86_ops_mem_enc_unreg_region(struct kvm *kvm, + struct kvm_enc_region *argp); +extern int kvm_x86_ops_get_msr_feature(struct kvm_msr_entry *entry); +extern int kvm_x86_ops_nested_enable_evmcs(struct kvm_vcpu *vcpu, + uint16_t *vmcs_version); +extern uint16_t kvm_x86_ops_nested_get_evmcs_version(struct kvm_vcpu *vcpu); +extern bool kvm_x86_ops_need_emulation_on_page_fault(struct kvm_vcpu *vcpu); +extern bool kvm_x86_ops_apic_init_signal_blocked(struct kvm_vcpu *vcpu); + +#endif /* _ASM_X86_KVM_OPS_H */ diff --git a/arch/x86/kvm/svm_ops.c b/arch/x86/kvm/svm_ops.c new file mode 100644 index 000000000000..2aaabda92179 --- /dev/null +++ b/arch/x86/kvm/svm_ops.c @@ -0,0 +1,672 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * arch/x86/kvm/svm_ops.c + * + * Copyright 2019 Red Hat, Inc. + */ + +int kvm_x86_ops_cpu_has_kvm_support(void) +{ + return has_svm(); +} + +int kvm_x86_ops_disabled_by_bios(void) +{ + return is_disabled(); +} + +int kvm_x86_ops_hardware_enable(void) +{ + return svm_hardware_enable(); +} + +void kvm_x86_ops_hardware_disable(void) +{ + svm_hardware_disable(); +} + +__init int kvm_x86_ops_check_processor_compatibility(void) +{ + return svm_check_processor_compat(); +} + +__init int kvm_x86_ops_hardware_setup(void) +{ + return svm_hardware_setup(); +} + +void kvm_x86_ops_hardware_unsetup(void) +{ + svm_hardware_unsetup(); +} + +bool kvm_x86_ops_cpu_has_accelerated_tpr(void) +{ + return svm_cpu_has_accelerated_tpr(); +} + +bool kvm_x86_ops_has_emulated_msr(int index) +{ + return svm_has_emulated_msr(index); +} + +void kvm_x86_ops_cpuid_update(struct kvm_vcpu *vcpu) +{ + svm_cpuid_update(vcpu); +} + +struct kvm *kvm_x86_ops_vm_alloc(void) +{ + return svm_vm_alloc(); +} + +void kvm_x86_ops_vm_free(struct kvm *kvm) +{ + svm_vm_free(kvm); +} + +int kvm_x86_ops_vm_init(struct kvm *kvm) +{ + return avic_vm_init(kvm); +} + +void kvm_x86_ops_vm_destroy(struct kvm *kvm) +{ + svm_vm_destroy(kvm); +} + +struct kvm_vcpu *kvm_x86_ops_vcpu_create(struct kvm *kvm, unsigned id) +{ + return svm_create_vcpu(kvm, id); +} + +void kvm_x86_ops_vcpu_free(struct kvm_vcpu *vcpu) +{ + svm_free_vcpu(vcpu); +} + +void kvm_x86_ops_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) +{ + svm_vcpu_reset(vcpu, init_event); +} + +void kvm_x86_ops_prepare_guest_switch(struct kvm_vcpu *vcpu) +{ + svm_prepare_guest_switch(vcpu); +} + +void kvm_x86_ops_vcpu_load(struct kvm_vcpu *vcpu, int cpu) +{ + svm_vcpu_load(vcpu, cpu); +} + +void kvm_x86_ops_vcpu_put(struct kvm_vcpu *vcpu) +{ + svm_vcpu_put(vcpu); +} + +void kvm_x86_ops_update_bp_intercept(struct kvm_vcpu *vcpu) +{ + update_bp_intercept(vcpu); +} + +int kvm_x86_ops_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) +{ + return svm_get_msr(vcpu, msr); +} + +int kvm_x86_ops_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) +{ + return svm_set_msr(vcpu, msr); +} + +u64 kvm_x86_ops_get_segment_base(struct kvm_vcpu *vcpu, int seg) +{ + return svm_get_segment_base(vcpu, seg); +} + +void kvm_x86_ops_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, + int seg) +{ + svm_get_segment(vcpu, var, seg); +} + +int kvm_x86_ops_get_cpl(struct kvm_vcpu *vcpu) +{ + return svm_get_cpl(vcpu); +} + +void kvm_x86_ops_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, + int seg) +{ + svm_set_segment(vcpu, var, seg); +} + +void kvm_x86_ops_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l) +{ + kvm_get_cs_db_l_bits(vcpu, db, l); +} + +void kvm_x86_ops_decache_cr0_guest_bits(struct kvm_vcpu *vcpu) +{ + svm_decache_cr0_guest_bits(vcpu); +} + +void kvm_x86_ops_decache_cr3(struct kvm_vcpu *vcpu) +{ + svm_decache_cr3(vcpu); +} + +void kvm_x86_ops_decache_cr4_guest_bits(struct kvm_vcpu *vcpu) +{ + svm_decache_cr4_guest_bits(vcpu); +} + +void kvm_x86_ops_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) +{ + svm_set_cr0(vcpu, cr0); +} + +void kvm_x86_ops_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) +{ + svm_set_cr3(vcpu, cr3); +} + +int kvm_x86_ops_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) +{ + return svm_set_cr4(vcpu, cr4); +} + +void kvm_x86_ops_set_efer(struct kvm_vcpu *vcpu, u64 efer) +{ + svm_set_efer(vcpu, efer); +} + +void kvm_x86_ops_get_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +{ + svm_get_idt(vcpu, dt); +} + +void kvm_x86_ops_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +{ + svm_set_idt(vcpu, dt); +} + +void kvm_x86_ops_get_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +{ + svm_get_gdt(vcpu, dt); +} + +void kvm_x86_ops_set_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +{ + svm_set_gdt(vcpu, dt); +} + +u64 kvm_x86_ops_get_dr6(struct kvm_vcpu *vcpu) +{ + return svm_get_dr6(vcpu); +} + +void kvm_x86_ops_set_dr6(struct kvm_vcpu *vcpu, unsigned long value) +{ + svm_set_dr6(vcpu, value); +} + +void kvm_x86_ops_sync_dirty_debug_regs(struct kvm_vcpu *vcpu) +{ + svm_sync_dirty_debug_regs(vcpu); +} + +void kvm_x86_ops_set_dr7(struct kvm_vcpu *vcpu, unsigned long value) +{ + svm_set_dr7(vcpu, value); +} + +void kvm_x86_ops_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) +{ + svm_cache_reg(vcpu, reg); +} + +unsigned long kvm_x86_ops_get_rflags(struct kvm_vcpu *vcpu) +{ + return svm_get_rflags(vcpu); +} + +void kvm_x86_ops_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags) +{ + svm_set_rflags(vcpu, rflags); +} + +void kvm_x86_ops_tlb_flush(struct kvm_vcpu *vcpu, bool invalidate_gpa) +{ + svm_flush_tlb(vcpu, invalidate_gpa); +} + +int kvm_x86_ops_tlb_remote_flush(struct kvm *kvm) +{ + return kvm_x86_ops->tlb_remote_flush(kvm); +} + +int kvm_x86_ops_tlb_remote_flush_with_range(struct kvm *kvm, + struct kvm_tlb_range *range) +{ + return kvm_x86_ops->tlb_remote_flush_with_range(kvm, range); +} + +void kvm_x86_ops_tlb_flush_gva(struct kvm_vcpu *vcpu, gva_t addr) +{ + svm_flush_tlb_gva(vcpu, addr); +} + +void kvm_x86_ops_run(struct kvm_vcpu *vcpu) +{ + svm_vcpu_run(vcpu); +} + +int kvm_x86_ops_handle_exit(struct kvm_vcpu *vcpu) +{ + return handle_exit(vcpu); +} + +int kvm_x86_ops_skip_emulated_instruction(struct kvm_vcpu *vcpu) +{ + return skip_emulated_instruction(vcpu); +} + +void kvm_x86_ops_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask) +{ + svm_set_interrupt_shadow(vcpu, mask); +} + +u32 kvm_x86_ops_get_interrupt_shadow(struct kvm_vcpu *vcpu) +{ + return svm_get_interrupt_shadow(vcpu); +} + +void kvm_x86_ops_patch_hypercall(struct kvm_vcpu *vcpu, + unsigned char *hypercall_addr) +{ + svm_patch_hypercall(vcpu, hypercall_addr); +} + +void kvm_x86_ops_set_irq(struct kvm_vcpu *vcpu) +{ + svm_set_irq(vcpu); +} + +void kvm_x86_ops_set_nmi(struct kvm_vcpu *vcpu) +{ + svm_inject_nmi(vcpu); +} + +void kvm_x86_ops_queue_exception(struct kvm_vcpu *vcpu) +{ + svm_queue_exception(vcpu); +} + +void kvm_x86_ops_cancel_injection(struct kvm_vcpu *vcpu) +{ + svm_cancel_injection(vcpu); +} + +int kvm_x86_ops_interrupt_allowed(struct kvm_vcpu *vcpu) +{ + return svm_interrupt_allowed(vcpu); +} + +int kvm_x86_ops_nmi_allowed(struct kvm_vcpu *vcpu) +{ + return svm_nmi_allowed(vcpu); +} + +bool kvm_x86_ops_get_nmi_mask(struct kvm_vcpu *vcpu) +{ + return svm_get_nmi_mask(vcpu); +} + +void kvm_x86_ops_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked) +{ + svm_set_nmi_mask(vcpu, masked); +} + +void kvm_x86_ops_enable_nmi_window(struct kvm_vcpu *vcpu) +{ + enable_nmi_window(vcpu); +} + +void kvm_x86_ops_enable_irq_window(struct kvm_vcpu *vcpu) +{ + enable_irq_window(vcpu); +} + +void kvm_x86_ops_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr) +{ + update_cr8_intercept(vcpu, tpr, irr); +} + +bool kvm_x86_ops_get_enable_apicv(struct kvm_vcpu *vcpu) +{ + return svm_get_enable_apicv(vcpu); +} + +void kvm_x86_ops_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu) +{ + svm_refresh_apicv_exec_ctrl(vcpu); +} + +void kvm_x86_ops_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr) +{ + svm_hwapic_irr_update(vcpu, max_irr); +} + +void kvm_x86_ops_hwapic_isr_update(struct kvm_vcpu *vcpu, int isr) +{ + svm_hwapic_isr_update(vcpu, isr); +} + +bool kvm_x86_ops_guest_apic_has_interrupt(struct kvm_vcpu *vcpu) +{ + return kvm_x86_ops->guest_apic_has_interrupt(vcpu); +} + +void kvm_x86_ops_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) +{ + svm_load_eoi_exitmap(vcpu, eoi_exit_bitmap); +} + +void kvm_x86_ops_set_virtual_apic_mode(struct kvm_vcpu *vcpu) +{ + svm_set_virtual_apic_mode(vcpu); +} + +void kvm_x86_ops_set_apic_access_page_addr(struct kvm_vcpu *vcpu, hpa_t hpa) +{ + kvm_x86_ops->set_apic_access_page_addr(vcpu, hpa); +} + +void kvm_x86_ops_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector) +{ + svm_deliver_avic_intr(vcpu, vector); +} + +int kvm_x86_ops_sync_pir_to_irr(struct kvm_vcpu *vcpu) +{ + return kvm_lapic_find_highest_irr(vcpu); +} + +int kvm_x86_ops_set_tss_addr(struct kvm *kvm, unsigned int addr) +{ + return svm_set_tss_addr(kvm, addr); +} + +int kvm_x86_ops_set_identity_map_addr(struct kvm *kvm, u64 ident_addr) +{ + return svm_set_identity_map_addr(kvm, ident_addr); +} + +int kvm_x86_ops_get_tdp_level(struct kvm_vcpu *vcpu) +{ + return get_npt_level(vcpu); +} + +u64 kvm_x86_ops_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) +{ + return svm_get_mt_mask(vcpu, gfn, is_mmio); +} + +int kvm_x86_ops_get_lpage_level(void) +{ + return svm_get_lpage_level(); +} + +bool kvm_x86_ops_rdtscp_supported(void) +{ + return svm_rdtscp_supported(); +} + +bool kvm_x86_ops_invpcid_supported(void) +{ + return svm_invpcid_supported(); +} + +void kvm_x86_ops_set_tdp_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) +{ + set_tdp_cr3(vcpu, cr3); +} + +void kvm_x86_ops_set_supported_cpuid(u32 func, struct kvm_cpuid_entry2 *entry) +{ + svm_set_supported_cpuid(func, entry); +} + +bool kvm_x86_ops_has_wbinvd_exit(void) +{ + return svm_has_wbinvd_exit(); +} + +u64 kvm_x86_ops_read_l1_tsc_offset(struct kvm_vcpu *vcpu) +{ + return svm_read_l1_tsc_offset(vcpu); +} + +u64 kvm_x86_ops_write_l1_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) +{ + return svm_write_l1_tsc_offset(vcpu, offset); +} + +void kvm_x86_ops_get_exit_info(struct kvm_vcpu *vcpu, u64 *info1, u64 *info2) +{ + svm_get_exit_info(vcpu, info1, info2); +} + +int kvm_x86_ops_check_intercept(struct kvm_vcpu *vcpu, + struct x86_instruction_info *info, + enum x86_intercept_stage stage) +{ + return svm_check_intercept(vcpu, info, stage); +} + +void kvm_x86_ops_handle_exit_irqoff(struct kvm_vcpu *vcpu) +{ + svm_handle_exit_irqoff(vcpu); +} + +bool kvm_x86_ops_mpx_supported(void) +{ + return svm_mpx_supported(); +} + +bool kvm_x86_ops_xsaves_supported(void) +{ + return svm_xsaves_supported(); +} + +bool kvm_x86_ops_umip_emulated(void) +{ + return svm_umip_emulated(); +} + +bool kvm_x86_ops_pt_supported(void) +{ + return svm_pt_supported(); +} + +int kvm_x86_ops_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr) +{ + return kvm_x86_ops->check_nested_events(vcpu, external_intr); +} + +void kvm_x86_ops_request_immediate_exit(struct kvm_vcpu *vcpu) +{ + __kvm_request_immediate_exit(vcpu); +} + +void kvm_x86_ops_sched_in(struct kvm_vcpu *kvm, int cpu) +{ + svm_sched_in(kvm, cpu); +} + +void kvm_x86_ops_slot_enable_log_dirty(struct kvm *kvm, + struct kvm_memory_slot *slot) +{ + kvm_x86_ops->slot_enable_log_dirty(kvm, slot); +} + +void kvm_x86_ops_slot_disable_log_dirty(struct kvm *kvm, + struct kvm_memory_slot *slot) +{ + kvm_x86_ops->slot_disable_log_dirty(kvm, slot); +} + +void kvm_x86_ops_flush_log_dirty(struct kvm *kvm) +{ + kvm_x86_ops->flush_log_dirty(kvm); +} + +void kvm_x86_ops_enable_log_dirty_pt_masked(struct kvm *kvm, + struct kvm_memory_slot *slot, + gfn_t offset, unsigned long mask) +{ + kvm_x86_ops->enable_log_dirty_pt_masked(kvm, slot, offset, mask); +} + +int kvm_x86_ops_write_log_dirty(struct kvm_vcpu *vcpu) +{ + return kvm_x86_ops->write_log_dirty(vcpu); +} + +int kvm_x86_ops_pre_block(struct kvm_vcpu *vcpu) +{ + return kvm_x86_ops->pre_block(vcpu); +} + +void kvm_x86_ops_post_block(struct kvm_vcpu *vcpu) +{ + kvm_x86_ops->post_block(vcpu); +} + +void kvm_x86_ops_vcpu_blocking(struct kvm_vcpu *vcpu) +{ + svm_vcpu_blocking(vcpu); +} + +void kvm_x86_ops_vcpu_unblocking(struct kvm_vcpu *vcpu) +{ + svm_vcpu_unblocking(vcpu); +} + +int kvm_x86_ops_update_pi_irte(struct kvm *kvm, unsigned int host_irq, + uint32_t guest_irq, bool set) +{ + return svm_update_pi_irte(kvm, host_irq, guest_irq, set); +} + +void kvm_x86_ops_apicv_post_state_restore(struct kvm_vcpu *vcpu) +{ + avic_post_state_restore(vcpu); +} + +bool kvm_x86_ops_dy_apicv_has_pending_interrupt(struct kvm_vcpu *vcpu) +{ + return svm_dy_apicv_has_pending_interrupt(vcpu); +} + +int kvm_x86_ops_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc, + bool *expired) +{ + return kvm_x86_ops->set_hv_timer(vcpu, guest_deadline_tsc, expired); +} + +void kvm_x86_ops_cancel_hv_timer(struct kvm_vcpu *vcpu) +{ + kvm_x86_ops->cancel_hv_timer(vcpu); +} + +void kvm_x86_ops_setup_mce(struct kvm_vcpu *vcpu) +{ + svm_setup_mce(vcpu); +} + +int kvm_x86_ops_get_nested_state(struct kvm_vcpu *vcpu, + struct kvm_nested_state __user *user_kvm_nested_state, + unsigned user_data_size) +{ + return kvm_x86_ops->get_nested_state(vcpu, user_kvm_nested_state, + user_data_size); +} + +int kvm_x86_ops_set_nested_state(struct kvm_vcpu *vcpu, + struct kvm_nested_state __user *user_kvm_nested_state, + struct kvm_nested_state *kvm_state) +{ + return kvm_x86_ops->set_nested_state(vcpu, user_kvm_nested_state, + kvm_state); +} + +void kvm_x86_ops_get_vmcs12_pages(struct kvm_vcpu *vcpu) +{ + kvm_x86_ops->get_vmcs12_pages(vcpu); +} + +int kvm_x86_ops_smi_allowed(struct kvm_vcpu *vcpu) +{ + return svm_smi_allowed(vcpu); +} + +int kvm_x86_ops_pre_enter_smm(struct kvm_vcpu *vcpu, char *smstate) +{ + return svm_pre_enter_smm(vcpu, smstate); +} + +int kvm_x86_ops_pre_leave_smm(struct kvm_vcpu *vcpu, const char *smstate) +{ + return svm_pre_leave_smm(vcpu, smstate); +} + +int kvm_x86_ops_enable_smi_window(struct kvm_vcpu *vcpu) +{ + return enable_smi_window(vcpu); +} + +int kvm_x86_ops_mem_enc_op(struct kvm *kvm, void __user *argp) +{ + return svm_mem_enc_op(kvm, argp); +} + +int kvm_x86_ops_mem_enc_reg_region(struct kvm *kvm, + struct kvm_enc_region *argp) +{ + return svm_register_enc_region(kvm, argp); +} + +int kvm_x86_ops_mem_enc_unreg_region(struct kvm *kvm, + struct kvm_enc_region *argp) +{ + return svm_unregister_enc_region(kvm, argp); +} + +int kvm_x86_ops_get_msr_feature(struct kvm_msr_entry *entry) +{ + return svm_get_msr_feature(entry); +} + +int kvm_x86_ops_nested_enable_evmcs(struct kvm_vcpu *vcpu, + uint16_t *vmcs_version) +{ + return nested_enable_evmcs(vcpu, vmcs_version); +} + +uint16_t kvm_x86_ops_nested_get_evmcs_version(struct kvm_vcpu *vcpu) +{ + return kvm_x86_ops->nested_get_evmcs_version(vcpu); +} + +bool kvm_x86_ops_need_emulation_on_page_fault(struct kvm_vcpu *vcpu) +{ + return svm_need_emulation_on_page_fault(vcpu); +} + +bool kvm_x86_ops_apic_init_signal_blocked(struct kvm_vcpu *vcpu) +{ + return svm_apic_init_signal_blocked(vcpu); +} diff --git a/arch/x86/kvm/vmx/vmx_ops.c b/arch/x86/kvm/vmx/vmx_ops.c new file mode 100644 index 000000000000..cdcad73935d9 --- /dev/null +++ b/arch/x86/kvm/vmx/vmx_ops.c @@ -0,0 +1,672 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * arch/x86/kvm/vmx/vmx_ops.c + * + * Copyright 2019 Red Hat, Inc. + */ + +__init int kvm_x86_ops_cpu_has_kvm_support(void) +{ + return cpu_has_kvm_support(); +} + +__init int kvm_x86_ops_disabled_by_bios(void) +{ + return vmx_disabled_by_bios(); +} + +int kvm_x86_ops_hardware_enable(void) +{ + return hardware_enable(); +} + +void kvm_x86_ops_hardware_disable(void) +{ + hardware_disable(); +} + +__init int kvm_x86_ops_check_processor_compatibility(void) +{ + return vmx_check_processor_compat(); +} + +__init int kvm_x86_ops_hardware_setup(void) +{ + return hardware_setup(); +} + +void kvm_x86_ops_hardware_unsetup(void) +{ + hardware_unsetup(); +} + +bool kvm_x86_ops_cpu_has_accelerated_tpr(void) +{ + return report_flexpriority(); +} + +bool kvm_x86_ops_has_emulated_msr(int index) +{ + return vmx_has_emulated_msr(index); +} + +void kvm_x86_ops_cpuid_update(struct kvm_vcpu *vcpu) +{ + vmx_cpuid_update(vcpu); +} + +struct kvm *kvm_x86_ops_vm_alloc(void) +{ + return vmx_vm_alloc(); +} + +void kvm_x86_ops_vm_free(struct kvm *kvm) +{ + vmx_vm_free(kvm); +} + +int kvm_x86_ops_vm_init(struct kvm *kvm) +{ + return vmx_vm_init(kvm); +} + +void kvm_x86_ops_vm_destroy(struct kvm *kvm) +{ + kvm_x86_ops->vm_destroy(kvm); +} + +struct kvm_vcpu *kvm_x86_ops_vcpu_create(struct kvm *kvm, unsigned id) +{ + return vmx_create_vcpu(kvm, id); +} + +void kvm_x86_ops_vcpu_free(struct kvm_vcpu *vcpu) +{ + vmx_free_vcpu(vcpu); +} + +void kvm_x86_ops_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) +{ + vmx_vcpu_reset(vcpu, init_event); +} + +void kvm_x86_ops_prepare_guest_switch(struct kvm_vcpu *vcpu) +{ + vmx_prepare_switch_to_guest(vcpu); +} + +void kvm_x86_ops_vcpu_load(struct kvm_vcpu *vcpu, int cpu) +{ + vmx_vcpu_load(vcpu, cpu); +} + +void kvm_x86_ops_vcpu_put(struct kvm_vcpu *vcpu) +{ + vmx_vcpu_put(vcpu); +} + +void kvm_x86_ops_update_bp_intercept(struct kvm_vcpu *vcpu) +{ + update_exception_bitmap(vcpu); +} + +int kvm_x86_ops_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) +{ + return vmx_get_msr(vcpu, msr); +} + +int kvm_x86_ops_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) +{ + return vmx_set_msr(vcpu, msr); +} + +u64 kvm_x86_ops_get_segment_base(struct kvm_vcpu *vcpu, int seg) +{ + return vmx_get_segment_base(vcpu, seg); +} + +void kvm_x86_ops_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, + int seg) +{ + vmx_get_segment(vcpu, var, seg); +} + +int kvm_x86_ops_get_cpl(struct kvm_vcpu *vcpu) +{ + return vmx_get_cpl(vcpu); +} + +void kvm_x86_ops_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, + int seg) +{ + vmx_set_segment(vcpu, var, seg); +} + +void kvm_x86_ops_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l) +{ + vmx_get_cs_db_l_bits(vcpu, db, l); +} + +void kvm_x86_ops_decache_cr0_guest_bits(struct kvm_vcpu *vcpu) +{ + vmx_decache_cr0_guest_bits(vcpu); +} + +void kvm_x86_ops_decache_cr3(struct kvm_vcpu *vcpu) +{ + vmx_decache_cr3(vcpu); +} + +void kvm_x86_ops_decache_cr4_guest_bits(struct kvm_vcpu *vcpu) +{ + vmx_decache_cr4_guest_bits(vcpu); +} + +void kvm_x86_ops_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) +{ + vmx_set_cr0(vcpu, cr0); +} + +void kvm_x86_ops_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) +{ + vmx_set_cr3(vcpu, cr3); +} + +int kvm_x86_ops_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) +{ + return vmx_set_cr4(vcpu, cr4); +} + +void kvm_x86_ops_set_efer(struct kvm_vcpu *vcpu, u64 efer) +{ + vmx_set_efer(vcpu, efer); +} + +void kvm_x86_ops_get_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +{ + vmx_get_idt(vcpu, dt); +} + +void kvm_x86_ops_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +{ + vmx_set_idt(vcpu, dt); +} + +void kvm_x86_ops_get_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +{ + vmx_get_gdt(vcpu, dt); +} + +void kvm_x86_ops_set_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +{ + vmx_set_gdt(vcpu, dt); +} + +u64 kvm_x86_ops_get_dr6(struct kvm_vcpu *vcpu) +{ + return vmx_get_dr6(vcpu); +} + +void kvm_x86_ops_set_dr6(struct kvm_vcpu *vcpu, unsigned long value) +{ + vmx_set_dr6(vcpu, value); +} + +void kvm_x86_ops_sync_dirty_debug_regs(struct kvm_vcpu *vcpu) +{ + vmx_sync_dirty_debug_regs(vcpu); +} + +void kvm_x86_ops_set_dr7(struct kvm_vcpu *vcpu, unsigned long value) +{ + vmx_set_dr7(vcpu, value); +} + +void kvm_x86_ops_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) +{ + vmx_cache_reg(vcpu, reg); +} + +unsigned long kvm_x86_ops_get_rflags(struct kvm_vcpu *vcpu) +{ + return vmx_get_rflags(vcpu); +} + +void kvm_x86_ops_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags) +{ + vmx_set_rflags(vcpu, rflags); +} + +void kvm_x86_ops_tlb_flush(struct kvm_vcpu *vcpu, bool invalidate_gpa) +{ + vmx_flush_tlb(vcpu, invalidate_gpa); +} + +int kvm_x86_ops_tlb_remote_flush(struct kvm *kvm) +{ + return kvm_x86_ops->tlb_remote_flush(kvm); +} + +int kvm_x86_ops_tlb_remote_flush_with_range(struct kvm *kvm, + struct kvm_tlb_range *range) +{ + return kvm_x86_ops->tlb_remote_flush_with_range(kvm, range); +} + +void kvm_x86_ops_tlb_flush_gva(struct kvm_vcpu *vcpu, gva_t addr) +{ + vmx_flush_tlb_gva(vcpu, addr); +} + +void kvm_x86_ops_run(struct kvm_vcpu *vcpu) +{ + vmx_vcpu_run(vcpu); +} + +int kvm_x86_ops_handle_exit(struct kvm_vcpu *vcpu) +{ + return vmx_handle_exit(vcpu); +} + +int kvm_x86_ops_skip_emulated_instruction(struct kvm_vcpu *vcpu) +{ + return __skip_emulated_instruction(vcpu); +} + +void kvm_x86_ops_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask) +{ + vmx_set_interrupt_shadow(vcpu, mask); +} + +u32 kvm_x86_ops_get_interrupt_shadow(struct kvm_vcpu *vcpu) +{ + return vmx_get_interrupt_shadow(vcpu); +} + +void kvm_x86_ops_patch_hypercall(struct kvm_vcpu *vcpu, + unsigned char *hypercall_addr) +{ + vmx_patch_hypercall(vcpu, hypercall_addr); +} + +void kvm_x86_ops_set_irq(struct kvm_vcpu *vcpu) +{ + vmx_inject_irq(vcpu); +} + +void kvm_x86_ops_set_nmi(struct kvm_vcpu *vcpu) +{ + vmx_inject_nmi(vcpu); +} + +void kvm_x86_ops_queue_exception(struct kvm_vcpu *vcpu) +{ + vmx_queue_exception(vcpu); +} + +void kvm_x86_ops_cancel_injection(struct kvm_vcpu *vcpu) +{ + vmx_cancel_injection(vcpu); +} + +int kvm_x86_ops_interrupt_allowed(struct kvm_vcpu *vcpu) +{ + return vmx_interrupt_allowed(vcpu); +} + +int kvm_x86_ops_nmi_allowed(struct kvm_vcpu *vcpu) +{ + return vmx_nmi_allowed(vcpu); +} + +bool kvm_x86_ops_get_nmi_mask(struct kvm_vcpu *vcpu) +{ + return vmx_get_nmi_mask(vcpu); +} + +void kvm_x86_ops_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked) +{ + vmx_set_nmi_mask(vcpu, masked); +} + +void kvm_x86_ops_enable_nmi_window(struct kvm_vcpu *vcpu) +{ + enable_nmi_window(vcpu); +} + +void kvm_x86_ops_enable_irq_window(struct kvm_vcpu *vcpu) +{ + enable_irq_window(vcpu); +} + +void kvm_x86_ops_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr) +{ + update_cr8_intercept(vcpu, tpr, irr); +} + +bool kvm_x86_ops_get_enable_apicv(struct kvm_vcpu *vcpu) +{ + return vmx_get_enable_apicv(vcpu); +} + +void kvm_x86_ops_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu) +{ + vmx_refresh_apicv_exec_ctrl(vcpu); +} + +void kvm_x86_ops_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr) +{ + vmx_hwapic_irr_update(vcpu, max_irr); +} + +void kvm_x86_ops_hwapic_isr_update(struct kvm_vcpu *vcpu, int isr) +{ + vmx_hwapic_isr_update(vcpu, isr); +} + +bool kvm_x86_ops_guest_apic_has_interrupt(struct kvm_vcpu *vcpu) +{ + return vmx_guest_apic_has_interrupt(vcpu); +} + +void kvm_x86_ops_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) +{ + vmx_load_eoi_exitmap(vcpu, eoi_exit_bitmap); +} + +void kvm_x86_ops_set_virtual_apic_mode(struct kvm_vcpu *vcpu) +{ + vmx_set_virtual_apic_mode(vcpu); +} + +void kvm_x86_ops_set_apic_access_page_addr(struct kvm_vcpu *vcpu, hpa_t hpa) +{ + vmx_set_apic_access_page_addr(vcpu, hpa); +} + +void kvm_x86_ops_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector) +{ + vmx_deliver_posted_interrupt(vcpu, vector); +} + +int kvm_x86_ops_sync_pir_to_irr(struct kvm_vcpu *vcpu) +{ + return vmx_sync_pir_to_irr(vcpu); +} + +int kvm_x86_ops_set_tss_addr(struct kvm *kvm, unsigned int addr) +{ + return vmx_set_tss_addr(kvm, addr); +} + +int kvm_x86_ops_set_identity_map_addr(struct kvm *kvm, u64 ident_addr) +{ + return vmx_set_identity_map_addr(kvm, ident_addr); +} + +int kvm_x86_ops_get_tdp_level(struct kvm_vcpu *vcpu) +{ + return get_ept_level(vcpu); +} + +u64 kvm_x86_ops_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) +{ + return vmx_get_mt_mask(vcpu, gfn, is_mmio); +} + +int kvm_x86_ops_get_lpage_level(void) +{ + return vmx_get_lpage_level(); +} + +bool kvm_x86_ops_rdtscp_supported(void) +{ + return vmx_rdtscp_supported(); +} + +bool kvm_x86_ops_invpcid_supported(void) +{ + return vmx_invpcid_supported(); +} + +void kvm_x86_ops_set_tdp_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) +{ + vmx_set_cr3(vcpu, cr3); +} + +void kvm_x86_ops_set_supported_cpuid(u32 func, struct kvm_cpuid_entry2 *entry) +{ + vmx_set_supported_cpuid(func, entry); +} + +bool kvm_x86_ops_has_wbinvd_exit(void) +{ + return cpu_has_vmx_wbinvd_exit(); +} + +u64 kvm_x86_ops_read_l1_tsc_offset(struct kvm_vcpu *vcpu) +{ + return vmx_read_l1_tsc_offset(vcpu); +} + +u64 kvm_x86_ops_write_l1_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) +{ + return vmx_write_l1_tsc_offset(vcpu, offset); +} + +void kvm_x86_ops_get_exit_info(struct kvm_vcpu *vcpu, u64 *info1, u64 *info2) +{ + vmx_get_exit_info(vcpu, info1, info2); +} + +int kvm_x86_ops_check_intercept(struct kvm_vcpu *vcpu, + struct x86_instruction_info *info, + enum x86_intercept_stage stage) +{ + return vmx_check_intercept(vcpu, info, stage); +} + +void kvm_x86_ops_handle_exit_irqoff(struct kvm_vcpu *vcpu) +{ + vmx_handle_exit_irqoff(vcpu); +} + +bool kvm_x86_ops_mpx_supported(void) +{ + return vmx_mpx_supported(); +} + +bool kvm_x86_ops_xsaves_supported(void) +{ + return vmx_xsaves_supported(); +} + +bool kvm_x86_ops_umip_emulated(void) +{ + return vmx_umip_emulated(); +} + +bool kvm_x86_ops_pt_supported(void) +{ + return vmx_pt_supported(); +} + +int kvm_x86_ops_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr) +{ + return kvm_x86_ops->check_nested_events(vcpu, external_intr); +} + +void kvm_x86_ops_request_immediate_exit(struct kvm_vcpu *vcpu) +{ + vmx_request_immediate_exit(vcpu); +} + +void kvm_x86_ops_sched_in(struct kvm_vcpu *kvm, int cpu) +{ + vmx_sched_in(kvm, cpu); +} + +void kvm_x86_ops_slot_enable_log_dirty(struct kvm *kvm, + struct kvm_memory_slot *slot) +{ + vmx_slot_enable_log_dirty(kvm, slot); +} + +void kvm_x86_ops_slot_disable_log_dirty(struct kvm *kvm, + struct kvm_memory_slot *slot) +{ + vmx_slot_disable_log_dirty(kvm, slot); +} + +void kvm_x86_ops_flush_log_dirty(struct kvm *kvm) +{ + vmx_flush_log_dirty(kvm); +} + +void kvm_x86_ops_enable_log_dirty_pt_masked(struct kvm *kvm, + struct kvm_memory_slot *slot, + gfn_t offset, unsigned long mask) +{ + vmx_enable_log_dirty_pt_masked(kvm, slot, offset, mask); +} + +int kvm_x86_ops_write_log_dirty(struct kvm_vcpu *vcpu) +{ + return vmx_write_pml_buffer(vcpu); +} + +int kvm_x86_ops_pre_block(struct kvm_vcpu *vcpu) +{ + return vmx_pre_block(vcpu); +} + +void kvm_x86_ops_post_block(struct kvm_vcpu *vcpu) +{ + vmx_post_block(vcpu); +} + +void kvm_x86_ops_vcpu_blocking(struct kvm_vcpu *vcpu) +{ + kvm_x86_ops->vcpu_blocking(vcpu); +} + +void kvm_x86_ops_vcpu_unblocking(struct kvm_vcpu *vcpu) +{ + kvm_x86_ops->vcpu_unblocking(vcpu); +} + +int kvm_x86_ops_update_pi_irte(struct kvm *kvm, unsigned int host_irq, + uint32_t guest_irq, bool set) +{ + return vmx_update_pi_irte(kvm, host_irq, guest_irq, set); +} + +void kvm_x86_ops_apicv_post_state_restore(struct kvm_vcpu *vcpu) +{ + vmx_apicv_post_state_restore(vcpu); +} + +bool kvm_x86_ops_dy_apicv_has_pending_interrupt(struct kvm_vcpu *vcpu) +{ + return vmx_dy_apicv_has_pending_interrupt(vcpu); +} + +int kvm_x86_ops_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc, + bool *expired) +{ + return vmx_set_hv_timer(vcpu, guest_deadline_tsc, expired); +} + +void kvm_x86_ops_cancel_hv_timer(struct kvm_vcpu *vcpu) +{ + vmx_cancel_hv_timer(vcpu); +} + +void kvm_x86_ops_setup_mce(struct kvm_vcpu *vcpu) +{ + vmx_setup_mce(vcpu); +} + +int kvm_x86_ops_get_nested_state(struct kvm_vcpu *vcpu, + struct kvm_nested_state __user *user_kvm_nested_state, + unsigned user_data_size) +{ + return kvm_x86_ops->get_nested_state(vcpu, user_kvm_nested_state, + user_data_size); +} + +int kvm_x86_ops_set_nested_state(struct kvm_vcpu *vcpu, + struct kvm_nested_state __user *user_kvm_nested_state, + struct kvm_nested_state *kvm_state) +{ + return kvm_x86_ops->set_nested_state(vcpu, user_kvm_nested_state, + kvm_state); +} + +void kvm_x86_ops_get_vmcs12_pages(struct kvm_vcpu *vcpu) +{ + kvm_x86_ops->get_vmcs12_pages(vcpu); +} + +int kvm_x86_ops_smi_allowed(struct kvm_vcpu *vcpu) +{ + return vmx_smi_allowed(vcpu); +} + +int kvm_x86_ops_pre_enter_smm(struct kvm_vcpu *vcpu, char *smstate) +{ + return vmx_pre_enter_smm(vcpu, smstate); +} + +int kvm_x86_ops_pre_leave_smm(struct kvm_vcpu *vcpu, const char *smstate) +{ + return vmx_pre_leave_smm(vcpu, smstate); +} + +int kvm_x86_ops_enable_smi_window(struct kvm_vcpu *vcpu) +{ + return enable_smi_window(vcpu); +} + +int kvm_x86_ops_mem_enc_op(struct kvm *kvm, void __user *argp) +{ + return kvm_x86_ops->mem_enc_op(kvm, argp); +} + +int kvm_x86_ops_mem_enc_reg_region(struct kvm *kvm, + struct kvm_enc_region *argp) +{ + return kvm_x86_ops->mem_enc_reg_region(kvm, argp); +} + +int kvm_x86_ops_mem_enc_unreg_region(struct kvm *kvm, + struct kvm_enc_region *argp) +{ + return kvm_x86_ops->mem_enc_unreg_region(kvm, argp); +} + +int kvm_x86_ops_get_msr_feature(struct kvm_msr_entry *entry) +{ + return vmx_get_msr_feature(entry); +} + +int kvm_x86_ops_nested_enable_evmcs(struct kvm_vcpu *vcpu, + uint16_t *vmcs_version) +{ + return kvm_x86_ops->nested_enable_evmcs(vcpu, vmcs_version); +} + +uint16_t kvm_x86_ops_nested_get_evmcs_version(struct kvm_vcpu *vcpu) +{ + return kvm_x86_ops->nested_get_evmcs_version(vcpu); +} + +bool kvm_x86_ops_need_emulation_on_page_fault(struct kvm_vcpu *vcpu) +{ + return vmx_need_emulation_on_page_fault(vcpu); +} + +bool kvm_x86_ops_apic_init_signal_blocked(struct kvm_vcpu *vcpu) +{ + return vmx_apic_init_signal_blocked(vcpu); +} From patchwork Fri Sep 20 21:24:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154889 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8162615E6 for ; Fri, 20 Sep 2019 21:26:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6A92C2086A for ; Fri, 20 Sep 2019 21:26:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730162AbfITVZM (ORCPT ); Fri, 20 Sep 2019 17:25:12 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51034 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730149AbfITVZM (ORCPT ); Fri, 20 Sep 2019 17:25:12 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 80A1C18C4270; Fri, 20 Sep 2019 21:25:12 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 239AA1001B2D; Fri, 20 Sep 2019 21:25:10 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 03/17] KVM: monolithic: x86: handle the request_immediate_exit variation Date: Fri, 20 Sep 2019 17:24:55 -0400 Message-Id: <20190920212509.2578-4-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.62]); Fri, 20 Sep 2019 21:25:12 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org request_immediate_exit is one of those few cases where the pointer to function of the method isn't fixed at build time and it requires special handling because hardware_setup() may override it at runtime. Signed-off-by: Andrea Arcangeli --- arch/x86/kvm/vmx/vmx_ops.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/vmx_ops.c b/arch/x86/kvm/vmx/vmx_ops.c index cdcad73935d9..25d441432901 100644 --- a/arch/x86/kvm/vmx/vmx_ops.c +++ b/arch/x86/kvm/vmx/vmx_ops.c @@ -498,7 +498,10 @@ int kvm_x86_ops_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr) void kvm_x86_ops_request_immediate_exit(struct kvm_vcpu *vcpu) { - vmx_request_immediate_exit(vcpu); + if (likely(enable_preemption_timer)) + vmx_request_immediate_exit(vcpu); + else + __kvm_request_immediate_exit(vcpu); } void kvm_x86_ops_sched_in(struct kvm_vcpu *kvm, int cpu) From patchwork Fri Sep 20 21:24:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154881 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9276615E6 for ; Fri, 20 Sep 2019 21:26:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6F95F20820 for ; Fri, 20 Sep 2019 21:26:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2394008AbfITV0Q (ORCPT ); Fri, 20 Sep 2019 17:26:16 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40482 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730151AbfITVZN (ORCPT ); Fri, 20 Sep 2019 17:25:13 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B089530833C1; Fri, 20 Sep 2019 21:25:12 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1868819C68; Fri, 20 Sep 2019 21:25:10 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 04/17] KVM: monolithic: x86: convert the kvm_pmu_ops methods to external functions Date: Fri, 20 Sep 2019 17:24:56 -0400 Message-Id: <20190920212509.2578-5-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Fri, 20 Sep 2019 21:25:12 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This replaces all kvm_pmu_ops pointer to functions with regular external functions that don't require indirect calls. Signed-off-by: Andrea Arcangeli --- arch/x86/kvm/pmu_amd_ops.c | 68 ++++++++++++++++++++++++++++++++ arch/x86/kvm/pmu_ops.h | 22 +++++++++++ arch/x86/kvm/vmx/pmu_intel_ops.c | 68 ++++++++++++++++++++++++++++++++ 3 files changed, 158 insertions(+) create mode 100644 arch/x86/kvm/pmu_amd_ops.c create mode 100644 arch/x86/kvm/pmu_ops.h create mode 100644 arch/x86/kvm/vmx/pmu_intel_ops.c diff --git a/arch/x86/kvm/pmu_amd_ops.c b/arch/x86/kvm/pmu_amd_ops.c new file mode 100644 index 000000000000..d2e9a244caa8 --- /dev/null +++ b/arch/x86/kvm/pmu_amd_ops.c @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * arch/x86/kvm/pmu_amd_ops.c + * + * Copyright 2019 Red Hat, Inc. + */ + +unsigned kvm_pmu_ops_find_arch_event(struct kvm_pmu *pmu, u8 event_select, + u8 unit_mask) +{ + return amd_find_arch_event(pmu, event_select, unit_mask); +} + +unsigned kvm_pmu_ops_find_fixed_event(int idx) +{ + return amd_find_fixed_event(idx); +} + +bool kvm_pmu_ops_pmc_is_enabled(struct kvm_pmc *pmc) +{ + return amd_pmc_is_enabled(pmc); +} + +struct kvm_pmc *kvm_pmu_ops_pmc_idx_to_pmc(struct kvm_pmu *pmu, int pmc_idx) +{ + return amd_pmc_idx_to_pmc(pmu, pmc_idx); +} + +struct kvm_pmc *kvm_pmu_ops_msr_idx_to_pmc(struct kvm_vcpu *vcpu, unsigned idx, + u64 *mask) +{ + return amd_msr_idx_to_pmc(vcpu, idx, mask); +} + +int kvm_pmu_ops_is_valid_msr_idx(struct kvm_vcpu *vcpu, unsigned idx) +{ + return amd_is_valid_msr_idx(vcpu, idx); +} + +bool kvm_pmu_ops_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr) +{ + return amd_is_valid_msr(vcpu, msr); +} + +int kvm_pmu_ops_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *data) +{ + return amd_pmu_get_msr(vcpu, msr, data); +} + +int kvm_pmu_ops_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) +{ + return amd_pmu_set_msr(vcpu, msr_info); +} + +void kvm_pmu_ops_refresh(struct kvm_vcpu *vcpu) +{ + amd_pmu_refresh(vcpu); +} + +void kvm_pmu_ops_init(struct kvm_vcpu *vcpu) +{ + amd_pmu_init(vcpu); +} + +void kvm_pmu_ops_reset(struct kvm_vcpu *vcpu) +{ + amd_pmu_reset(vcpu); +} diff --git a/arch/x86/kvm/pmu_ops.h b/arch/x86/kvm/pmu_ops.h new file mode 100644 index 000000000000..6230ce300cbe --- /dev/null +++ b/arch/x86/kvm/pmu_ops.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __KVM_X86_PMU_OPS_H +#define __KVM_X86_PMU_OPS_H + +extern unsigned kvm_pmu_ops_find_arch_event(struct kvm_pmu *pmu, + u8 event_select, u8 unit_mask); +extern unsigned kvm_pmu_ops_find_fixed_event(int idx); +extern bool kvm_pmu_ops_pmc_is_enabled(struct kvm_pmc *pmc); +extern struct kvm_pmc *kvm_pmu_ops_pmc_idx_to_pmc(struct kvm_pmu *pmu, + int pmc_idx); +extern struct kvm_pmc *kvm_pmu_ops_msr_idx_to_pmc(struct kvm_vcpu *vcpu, + unsigned idx, u64 *mask); +extern int kvm_pmu_ops_is_valid_msr_idx(struct kvm_vcpu *vcpu, unsigned idx); +extern bool kvm_pmu_ops_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr); +extern int kvm_pmu_ops_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *data); +extern int kvm_pmu_ops_set_msr(struct kvm_vcpu *vcpu, + struct msr_data *msr_info); +extern void kvm_pmu_ops_refresh(struct kvm_vcpu *vcpu); +extern void kvm_pmu_ops_init(struct kvm_vcpu *vcpu); +extern void kvm_pmu_ops_reset(struct kvm_vcpu *vcpu); + +#endif /* __KVM_X86_PMU_OPS_H */ diff --git a/arch/x86/kvm/vmx/pmu_intel_ops.c b/arch/x86/kvm/vmx/pmu_intel_ops.c new file mode 100644 index 000000000000..39f1b4af85e3 --- /dev/null +++ b/arch/x86/kvm/vmx/pmu_intel_ops.c @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * arch/x86/kvm/vmx/pmu_intel_ops.c + * + * Copyright 2019 Red Hat, Inc. + */ + +unsigned kvm_pmu_ops_find_arch_event(struct kvm_pmu *pmu, u8 event_select, + u8 unit_mask) +{ + return intel_find_arch_event(pmu, event_select, unit_mask); +} + +unsigned kvm_pmu_ops_find_fixed_event(int idx) +{ + return intel_find_fixed_event(idx); +} + +bool kvm_pmu_ops_pmc_is_enabled(struct kvm_pmc *pmc) +{ + return intel_pmc_is_enabled(pmc); +} + +struct kvm_pmc *kvm_pmu_ops_pmc_idx_to_pmc(struct kvm_pmu *pmu, int pmc_idx) +{ + return intel_pmc_idx_to_pmc(pmu, pmc_idx); +} + +struct kvm_pmc *kvm_pmu_ops_msr_idx_to_pmc(struct kvm_vcpu *vcpu, unsigned idx, + u64 *mask) +{ + return intel_msr_idx_to_pmc(vcpu, idx, mask); +} + +int kvm_pmu_ops_is_valid_msr_idx(struct kvm_vcpu *vcpu, unsigned idx) +{ + return intel_is_valid_msr_idx(vcpu, idx); +} + +bool kvm_pmu_ops_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr) +{ + return intel_is_valid_msr(vcpu, msr); +} + +int kvm_pmu_ops_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *data) +{ + return intel_pmu_get_msr(vcpu, msr, data); +} + +int kvm_pmu_ops_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) +{ + return intel_pmu_set_msr(vcpu, msr_info); +} + +void kvm_pmu_ops_refresh(struct kvm_vcpu *vcpu) +{ + intel_pmu_refresh(vcpu); +} + +void kvm_pmu_ops_init(struct kvm_vcpu *vcpu) +{ + intel_pmu_init(vcpu); +} + +void kvm_pmu_ops_reset(struct kvm_vcpu *vcpu) +{ + intel_pmu_reset(vcpu); +} From patchwork Fri Sep 20 21:24:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154877 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD3F915E6 for ; Fri, 20 Sep 2019 21:26:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BA1F42086A for ; Fri, 20 Sep 2019 21:26:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393609AbfITVZN (ORCPT ); Fri, 20 Sep 2019 17:25:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40484 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392045AbfITVZN (ORCPT ); Fri, 20 Sep 2019 17:25:13 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 61DB530832C0; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3EA615D6B2; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 05/17] KVM: monolithic: x86: enable the kvm_x86_ops external functions Date: Fri, 20 Sep 2019 17:24:57 -0400 Message-Id: <20190920212509.2578-6-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Fri, 20 Sep 2019 21:25:13 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Plug in the new external functions and their extern declarations in the respective kernel modules (kvm-intel and kvm-amd). Signed-off-by: Andrea Arcangeli --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/svm.c | 2 ++ arch/x86/kvm/vmx/vmx.c | 2 ++ 3 files changed, 6 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index a3a3ec73fa2f..99b30affb8ad 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1002,6 +1002,8 @@ struct kvm_lapic_irq { bool msi_redir_hint; }; +#include + struct kvm_x86_ops { int (*cpu_has_kvm_support)(void); /* __init */ int (*disabled_by_bios)(void); /* __init */ diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index 04fe21849b6e..3041abbd643b 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -7179,6 +7179,8 @@ static bool svm_apic_init_signal_blocked(struct kvm_vcpu *vcpu) (svm->vmcb->control.intercept & (1ULL << INTERCEPT_INIT)); } +#include "svm_ops.c" + static struct kvm_x86_ops svm_x86_ops __ro_after_init = { .cpu_has_kvm_support = has_svm, .disabled_by_bios = is_disabled, diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 4a99be1fae4e..9fc3969f6443 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7650,6 +7650,8 @@ static __exit void hardware_unsetup(void) free_kvm_area(); } +#include "vmx_ops.c" + static struct kvm_x86_ops vmx_x86_ops __ro_after_init = { .cpu_has_kvm_support = cpu_has_kvm_support, .disabled_by_bios = vmx_disabled_by_bios, From patchwork Fri Sep 20 21:24:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154879 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D49171599 for ; Fri, 20 Sep 2019 21:26:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BDCDE218AE for ; Fri, 20 Sep 2019 21:26:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2394017AbfITV0R (ORCPT ); Fri, 20 Sep 2019 17:26:17 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37392 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2391072AbfITVZN (ORCPT ); Fri, 20 Sep 2019 17:25:13 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 32F81308123B; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 15B945D6B2; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 06/17] KVM: monolithic: x86: enable the kvm_pmu_ops external functions Date: Fri, 20 Sep 2019 17:24:58 -0400 Message-Id: <20190920212509.2578-7-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Fri, 20 Sep 2019 21:25:13 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Plug in the new external functions and their extern declarations in the respective kernel modules (kvm-intel and kvm-amd). Signed-off-by: Andrea Arcangeli --- arch/x86/kvm/pmu.h | 2 ++ arch/x86/kvm/pmu_amd.c | 2 ++ arch/x86/kvm/vmx/pmu_intel.c | 2 ++ 3 files changed, 6 insertions(+) diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index 58265f761c3b..a9c2f68a40cc 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -19,6 +19,8 @@ struct kvm_event_hw_type_mapping { unsigned event_type; }; +#include "pmu_ops.h" + struct kvm_pmu_ops { unsigned (*find_arch_event)(struct kvm_pmu *pmu, u8 event_select, u8 unit_mask); diff --git a/arch/x86/kvm/pmu_amd.c b/arch/x86/kvm/pmu_amd.c index c8388389a3b0..12d1fa3ba35a 100644 --- a/arch/x86/kvm/pmu_amd.c +++ b/arch/x86/kvm/pmu_amd.c @@ -301,6 +301,8 @@ static void amd_pmu_reset(struct kvm_vcpu *vcpu) } } +#include "pmu_amd_ops.c" + struct kvm_pmu_ops amd_pmu_ops = { .find_arch_event = amd_find_arch_event, .find_fixed_event = amd_find_fixed_event, diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 4dea0e0e7e39..5ef36a75c31e 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -358,6 +358,8 @@ static void intel_pmu_reset(struct kvm_vcpu *vcpu) pmu->global_ovf_ctrl = 0; } +#include "pmu_intel_ops.c" + struct kvm_pmu_ops intel_pmu_ops = { .find_arch_event = intel_find_arch_event, .find_fixed_event = intel_find_fixed_event, From patchwork Fri Sep 20 21:24:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154855 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 318B51599 for ; Fri, 20 Sep 2019 21:25:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 19F34208C3 for ; Fri, 20 Sep 2019 21:25:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393817AbfITVZR (ORCPT ); Fri, 20 Sep 2019 17:25:17 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57066 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393802AbfITVZQ (ORCPT ); Fri, 20 Sep 2019 17:25:16 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CD8DCC054C52; Fri, 20 Sep 2019 21:25:15 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 80AAB60BF4; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 07/17] KVM: monolithic: x86: adjust the section prefixes Date: Fri, 20 Sep 2019 17:24:59 -0400 Message-Id: <20190920212509.2578-8-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Fri, 20 Sep 2019 21:25:15 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Adjusts the section prefixes of some KVM common code function because with the monolithic methods the section checker can now do a more accurate analysis at build time and this allows to build without CONFIG_SECTION_MISMATCH_WARN_ONLY=n. Signed-off-by: Andrea Arcangeli --- arch/x86/kvm/svm.c | 2 +- arch/x86/kvm/vmx/vmx.c | 2 +- arch/x86/kvm/x86.c | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index 3041abbd643b..5a48beb58083 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -1407,7 +1407,7 @@ static __init int svm_hardware_setup(void) return r; } -static __exit void svm_hardware_unsetup(void) +static void svm_hardware_unsetup(void) { int cpu; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 9fc3969f6443..09e6a477e06f 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7642,7 +7642,7 @@ static __init int hardware_setup(void) return r; } -static __exit void hardware_unsetup(void) +static void hardware_unsetup(void) { if (nested) nested_vmx_hardware_unsetup(); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index dfd641243568..c04894b61384 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9206,7 +9206,7 @@ void kvm_arch_hardware_disable(void) drop_user_return_notifiers(); } -int kvm_arch_hardware_setup(void) +__init int kvm_arch_hardware_setup(void) { int r; @@ -9237,7 +9237,7 @@ void kvm_arch_hardware_unsetup(void) kvm_x86_ops->hardware_unsetup(); } -int kvm_arch_check_processor_compat(void) +__init int kvm_arch_check_processor_compat(void) { return kvm_x86_ops->check_processor_compatibility(); } From patchwork Fri Sep 20 21:25:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154867 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A94491599 for ; Fri, 20 Sep 2019 21:25:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 875D52173E for ; Fri, 20 Sep 2019 21:25:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393793AbfITVZO (ORCPT ); Fri, 20 Sep 2019 17:25:14 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33544 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393109AbfITVZN (ORCPT ); Fri, 20 Sep 2019 17:25:13 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A881C8553B; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 853BE19C68; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 08/17] KVM: monolithic: adjust the section prefixes in the KVM common code Date: Fri, 20 Sep 2019 17:25:00 -0400 Message-Id: <20190920212509.2578-9-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 20 Sep 2019 21:25:13 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Adjusts the section prefixes of some KVM common code function because with the monolithic methods the section checker can now do a more accurate analysis at build time and this allows to build without CONFIG_SECTION_MISMATCH_WARN_ONLY=n. Signed-off-by: Andrea Arcangeli --- virt/kvm/kvm_main.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index e6de3159e682..9aa448ea688f 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -4235,13 +4235,13 @@ static void kvm_sched_out(struct preempt_notifier *pn, kvm_arch_vcpu_put(vcpu); } -static void check_processor_compat(void *rtn) +static __init void check_processor_compat(void *rtn) { *(int *)rtn = kvm_arch_check_processor_compat(); } -int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align, - struct module *module) +__init int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align, + struct module *module) { int r; int cpu; From patchwork Fri Sep 20 21:25:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154861 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2625C1599 for ; Fri, 20 Sep 2019 21:25:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 06F58217F5 for ; Fri, 20 Sep 2019 21:25:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393824AbfITVZR (ORCPT ); Fri, 20 Sep 2019 17:25:17 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59086 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393803AbfITVZP (ORCPT ); Fri, 20 Sep 2019 17:25:15 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CC9175D66B; Fri, 20 Sep 2019 21:25:15 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 421EB60BE2; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 09/17] KVM: monolithic: x86: remove kvm.ko Date: Fri, 20 Sep 2019 17:25:01 -0400 Message-Id: <20190920212509.2578-10-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Fri, 20 Sep 2019 21:25:15 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This removes kvm.ko and it links and duplicates all kvm.ko objects to both kvm-amd and kvm-intel. Signed-off-by: Andrea Arcangeli --- arch/x86/kvm/Makefile | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile index 31ecf7a76d5a..68b81f381369 100644 --- a/arch/x86/kvm/Makefile +++ b/arch/x86/kvm/Makefile @@ -12,9 +12,8 @@ kvm-y += x86.o mmu.o emulate.o i8259.o irq.o lapic.o \ i8254.o ioapic.o irq_comm.o cpuid.o pmu.o mtrr.o \ hyperv.o page_track.o debugfs.o -kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o vmx/evmcs.o vmx/nested.o -kvm-amd-y += svm.o pmu_amd.o +kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o vmx/evmcs.o vmx/nested.o $(kvm-y) +kvm-amd-y += svm.o pmu_amd.o $(kvm-y) -obj-$(CONFIG_KVM) += kvm.o obj-$(CONFIG_KVM_INTEL) += kvm-intel.o obj-$(CONFIG_KVM_AMD) += kvm-amd.o From patchwork Fri Sep 20 21:25:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154863 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 11D5115E6 for ; Fri, 20 Sep 2019 21:25:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BB41F20C01 for ; Fri, 20 Sep 2019 21:25:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393862AbfITVZk (ORCPT ); Fri, 20 Sep 2019 17:25:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46016 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393806AbfITVZR (ORCPT ); Fri, 20 Sep 2019 17:25:17 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1DBE018CB8EF; Fri, 20 Sep 2019 21:25:16 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2029E60BE0; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 10/17] KVM: monolithic: x86: use the external functions instead of kvm_x86_ops Date: Fri, 20 Sep 2019 17:25:02 -0400 Message-Id: <20190920212509.2578-11-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.63]); Fri, 20 Sep 2019 21:25:16 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Now that the new methods are plugged in and they are functional use them instead of invoking the pointer to functions through kvm_x86_ops. Signed-off-by: Andrea Arcangeli --- arch/x86/include/asm/kvm_host.h | 10 +- arch/x86/kvm/cpuid.c | 22 +-- arch/x86/kvm/hyperv.c | 6 +- arch/x86/kvm/kvm_cache_regs.h | 10 +- arch/x86/kvm/lapic.c | 28 +-- arch/x86/kvm/mmu.c | 26 +-- arch/x86/kvm/mmu.h | 4 +- arch/x86/kvm/pmu.c | 24 +-- arch/x86/kvm/pmu.h | 2 +- arch/x86/kvm/trace.h | 4 +- arch/x86/kvm/vmx/pmu_intel.c | 2 +- arch/x86/kvm/x86.c | 304 ++++++++++++++++---------------- arch/x86/kvm/x86.h | 2 +- 13 files changed, 222 insertions(+), 222 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 99b30affb8ad..bd5f4f900288 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1230,19 +1230,19 @@ extern struct kmem_cache *x86_fpu_cache; #define __KVM_HAVE_ARCH_VM_ALLOC static inline struct kvm *kvm_arch_alloc_vm(void) { - return kvm_x86_ops->vm_alloc(); + return kvm_x86_ops_vm_alloc(); } static inline void kvm_arch_free_vm(struct kvm *kvm) { - return kvm_x86_ops->vm_free(kvm); + return kvm_x86_ops_vm_free(kvm); } #define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLB static inline int kvm_arch_flush_remote_tlb(struct kvm *kvm) { if (kvm_x86_ops->tlb_remote_flush && - !kvm_x86_ops->tlb_remote_flush(kvm)) + !kvm_x86_ops_tlb_remote_flush(kvm)) return 0; else return -ENOTSUPP; @@ -1599,13 +1599,13 @@ static inline bool kvm_irq_is_postable(struct kvm_lapic_irq *irq) static inline void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) { if (kvm_x86_ops->vcpu_blocking) - kvm_x86_ops->vcpu_blocking(vcpu); + kvm_x86_ops_vcpu_blocking(vcpu); } static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) { if (kvm_x86_ops->vcpu_unblocking) - kvm_x86_ops->vcpu_unblocking(vcpu); + kvm_x86_ops_vcpu_unblocking(vcpu); } static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {} diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index dd5985eb61b4..d8d840fc703e 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -48,7 +48,7 @@ static u32 xstate_required_size(u64 xstate_bv, bool compacted) bool kvm_mpx_supported(void) { return ((host_xcr0 & (XFEATURE_MASK_BNDREGS | XFEATURE_MASK_BNDCSR)) - && kvm_x86_ops->mpx_supported()); + && kvm_x86_ops_mpx_supported()); } EXPORT_SYMBOL_GPL(kvm_mpx_supported); @@ -232,7 +232,7 @@ int kvm_vcpu_ioctl_set_cpuid(struct kvm_vcpu *vcpu, vcpu->arch.cpuid_nent = cpuid->nent; cpuid_fix_nx_cap(vcpu); kvm_apic_set_version(vcpu); - kvm_x86_ops->cpuid_update(vcpu); + kvm_x86_ops_cpuid_update(vcpu); r = kvm_update_cpuid(vcpu); out: @@ -255,7 +255,7 @@ int kvm_vcpu_ioctl_set_cpuid2(struct kvm_vcpu *vcpu, goto out; vcpu->arch.cpuid_nent = cpuid->nent; kvm_apic_set_version(vcpu); - kvm_x86_ops->cpuid_update(vcpu); + kvm_x86_ops_cpuid_update(vcpu); r = kvm_update_cpuid(vcpu); out: return r; @@ -341,10 +341,10 @@ static int __do_cpuid_func_emulated(struct kvm_cpuid_entry2 *entry, static inline void do_cpuid_7_mask(struct kvm_cpuid_entry2 *entry, int index) { - unsigned f_invpcid = kvm_x86_ops->invpcid_supported() ? F(INVPCID) : 0; + unsigned f_invpcid = kvm_x86_ops_invpcid_supported() ? F(INVPCID) : 0; unsigned f_mpx = kvm_mpx_supported() ? F(MPX) : 0; - unsigned f_umip = kvm_x86_ops->umip_emulated() ? F(UMIP) : 0; - unsigned f_intel_pt = kvm_x86_ops->pt_supported() ? F(INTEL_PT) : 0; + unsigned f_umip = kvm_x86_ops_umip_emulated() ? F(UMIP) : 0; + unsigned f_intel_pt = kvm_x86_ops_pt_supported() ? F(INTEL_PT) : 0; unsigned f_la57; /* cpuid 7.0.ebx */ @@ -426,16 +426,16 @@ static inline int __do_cpuid_func(struct kvm_cpuid_entry2 *entry, u32 function, int r; unsigned f_nx = is_efer_nx() ? F(NX) : 0; #ifdef CONFIG_X86_64 - unsigned f_gbpages = (kvm_x86_ops->get_lpage_level() == PT_PDPE_LEVEL) + unsigned f_gbpages = (kvm_x86_ops_get_lpage_level() == PT_PDPE_LEVEL) ? F(GBPAGES) : 0; unsigned f_lm = F(LM); #else unsigned f_gbpages = 0; unsigned f_lm = 0; #endif - unsigned f_rdtscp = kvm_x86_ops->rdtscp_supported() ? F(RDTSCP) : 0; - unsigned f_xsaves = kvm_x86_ops->xsaves_supported() ? F(XSAVES) : 0; - unsigned f_intel_pt = kvm_x86_ops->pt_supported() ? F(INTEL_PT) : 0; + unsigned f_rdtscp = kvm_x86_ops_rdtscp_supported() ? F(RDTSCP) : 0; + unsigned f_xsaves = kvm_x86_ops_xsaves_supported() ? F(XSAVES) : 0; + unsigned f_intel_pt = kvm_x86_ops_pt_supported() ? F(INTEL_PT) : 0; /* cpuid 1.edx */ const u32 kvm_cpuid_1_edx_x86_features = @@ -786,7 +786,7 @@ static inline int __do_cpuid_func(struct kvm_cpuid_entry2 *entry, u32 function, break; } - kvm_x86_ops->set_supported_cpuid(function, entry); + kvm_x86_ops_set_supported_cpuid(function, entry); r = 0; diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index fff790a3f4ee..e5177c5c126c 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -1015,7 +1015,7 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data, addr = gfn_to_hva(kvm, gfn); if (kvm_is_error_hva(addr)) return 1; - kvm_x86_ops->patch_hypercall(vcpu, instructions); + kvm_x86_ops_patch_hypercall(vcpu, instructions); ((unsigned char *)instructions)[3] = 0xc3; /* ret */ if (__copy_to_user((void __user *)addr, instructions, 4)) return 1; @@ -1600,7 +1600,7 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) * hypercall generates UD from non zero cpl and real mode * per HYPER-V spec */ - if (kvm_x86_ops->get_cpl(vcpu) != 0 || !is_protmode(vcpu)) { + if (kvm_x86_ops_get_cpl(vcpu) != 0 || !is_protmode(vcpu)) { kvm_queue_exception(vcpu, UD_VECTOR); return 1; } @@ -1794,7 +1794,7 @@ int kvm_vcpu_ioctl_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid, int i, nent = ARRAY_SIZE(cpuid_entries); if (kvm_x86_ops->nested_get_evmcs_version) - evmcs_ver = kvm_x86_ops->nested_get_evmcs_version(vcpu); + evmcs_ver = kvm_x86_ops_nested_get_evmcs_version(vcpu); /* Skip NESTED_FEATURES if eVMCS is not supported */ if (!evmcs_ver) diff --git a/arch/x86/kvm/kvm_cache_regs.h b/arch/x86/kvm/kvm_cache_regs.h index 1cc6c47dc77e..a8e80edd93cb 100644 --- a/arch/x86/kvm/kvm_cache_regs.h +++ b/arch/x86/kvm/kvm_cache_regs.h @@ -41,7 +41,7 @@ static inline unsigned long kvm_register_read(struct kvm_vcpu *vcpu, enum kvm_reg reg) { if (!test_bit(reg, (unsigned long *)&vcpu->arch.regs_avail)) - kvm_x86_ops->cache_reg(vcpu, reg); + kvm_x86_ops_cache_reg(vcpu, reg); return vcpu->arch.regs[reg]; } @@ -81,7 +81,7 @@ static inline u64 kvm_pdptr_read(struct kvm_vcpu *vcpu, int index) if (!test_bit(VCPU_EXREG_PDPTR, (unsigned long *)&vcpu->arch.regs_avail)) - kvm_x86_ops->cache_reg(vcpu, (enum kvm_reg)VCPU_EXREG_PDPTR); + kvm_x86_ops_cache_reg(vcpu, (enum kvm_reg)VCPU_EXREG_PDPTR); return vcpu->arch.walk_mmu->pdptrs[index]; } @@ -90,7 +90,7 @@ static inline ulong kvm_read_cr0_bits(struct kvm_vcpu *vcpu, ulong mask) { ulong tmask = mask & KVM_POSSIBLE_CR0_GUEST_BITS; if (tmask & vcpu->arch.cr0_guest_owned_bits) - kvm_x86_ops->decache_cr0_guest_bits(vcpu); + kvm_x86_ops_decache_cr0_guest_bits(vcpu); return vcpu->arch.cr0 & mask; } @@ -103,14 +103,14 @@ static inline ulong kvm_read_cr4_bits(struct kvm_vcpu *vcpu, ulong mask) { ulong tmask = mask & KVM_POSSIBLE_CR4_GUEST_BITS; if (tmask & vcpu->arch.cr4_guest_owned_bits) - kvm_x86_ops->decache_cr4_guest_bits(vcpu); + kvm_x86_ops_decache_cr4_guest_bits(vcpu); return vcpu->arch.cr4 & mask; } static inline ulong kvm_read_cr3(struct kvm_vcpu *vcpu) { if (!test_bit(VCPU_EXREG_CR3, (ulong *)&vcpu->arch.regs_avail)) - kvm_x86_ops->decache_cr3(vcpu); + kvm_x86_ops_decache_cr3(vcpu); return vcpu->arch.cr3; } diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 8675458c2205..d8dce6d40174 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -453,7 +453,7 @@ static inline void apic_clear_irr(int vec, struct kvm_lapic *apic) if (unlikely(vcpu->arch.apicv_active)) { /* need to update RVI */ kvm_lapic_clear_vector(vec, apic->regs + APIC_IRR); - kvm_x86_ops->hwapic_irr_update(vcpu, + kvm_x86_ops_hwapic_irr_update(vcpu, apic_find_highest_irr(apic)); } else { apic->irr_pending = false; @@ -478,7 +478,7 @@ static inline void apic_set_isr(int vec, struct kvm_lapic *apic) * just set SVI. */ if (unlikely(vcpu->arch.apicv_active)) - kvm_x86_ops->hwapic_isr_update(vcpu, vec); + kvm_x86_ops_hwapic_isr_update(vcpu, vec); else { ++apic->isr_count; BUG_ON(apic->isr_count > MAX_APIC_VECTOR); @@ -526,7 +526,7 @@ static inline void apic_clear_isr(int vec, struct kvm_lapic *apic) * and must be left alone. */ if (unlikely(vcpu->arch.apicv_active)) - kvm_x86_ops->hwapic_isr_update(vcpu, + kvm_x86_ops_hwapic_isr_update(vcpu, apic_find_highest_isr(apic)); else { --apic->isr_count; @@ -669,7 +669,7 @@ static int apic_has_interrupt_for_ppr(struct kvm_lapic *apic, u32 ppr) { int highest_irr; if (apic->vcpu->arch.apicv_active) - highest_irr = kvm_x86_ops->sync_pir_to_irr(apic->vcpu); + highest_irr = kvm_x86_ops_sync_pir_to_irr(apic->vcpu); else highest_irr = apic_find_highest_irr(apic); if (highest_irr == -1 || (highest_irr & 0xF0) <= ppr) @@ -1059,7 +1059,7 @@ static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode, } if (vcpu->arch.apicv_active) - kvm_x86_ops->deliver_posted_interrupt(vcpu, vector); + kvm_x86_ops_deliver_posted_interrupt(vcpu, vector); else { kvm_lapic_set_irr(vector, apic); @@ -1704,7 +1704,7 @@ static void cancel_hv_timer(struct kvm_lapic *apic) { WARN_ON(preemptible()); WARN_ON(!apic->lapic_timer.hv_timer_in_use); - kvm_x86_ops->cancel_hv_timer(apic->vcpu); + kvm_x86_ops_cancel_hv_timer(apic->vcpu); apic->lapic_timer.hv_timer_in_use = false; } @@ -1721,7 +1721,7 @@ static bool start_hv_timer(struct kvm_lapic *apic) if (!ktimer->tscdeadline) return false; - if (kvm_x86_ops->set_hv_timer(vcpu, ktimer->tscdeadline, &expired)) + if (kvm_x86_ops_set_hv_timer(vcpu, ktimer->tscdeadline, &expired)) return false; ktimer->hv_timer_in_use = true; @@ -2141,7 +2141,7 @@ void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value) kvm_apic_set_x2apic_id(apic, vcpu->vcpu_id); if ((old_value ^ value) & (MSR_IA32_APICBASE_ENABLE | X2APIC_ENABLE)) - kvm_x86_ops->set_virtual_apic_mode(vcpu); + kvm_x86_ops_set_virtual_apic_mode(vcpu); apic->base_address = apic->vcpu->arch.apic_base & MSR_IA32_APICBASE_BASE; @@ -2204,9 +2204,9 @@ void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event) vcpu->arch.pv_eoi.msr_val = 0; apic_update_ppr(apic); if (vcpu->arch.apicv_active) { - kvm_x86_ops->apicv_post_state_restore(vcpu); - kvm_x86_ops->hwapic_irr_update(vcpu, -1); - kvm_x86_ops->hwapic_isr_update(vcpu, -1); + kvm_x86_ops_apicv_post_state_restore(vcpu); + kvm_x86_ops_hwapic_irr_update(vcpu, -1); + kvm_x86_ops_hwapic_isr_update(vcpu, -1); } vcpu->arch.apic_arb_prio = 0; @@ -2458,10 +2458,10 @@ int kvm_apic_set_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s) 1 : count_vectors(apic->regs + APIC_ISR); apic->highest_isr_cache = -1; if (vcpu->arch.apicv_active) { - kvm_x86_ops->apicv_post_state_restore(vcpu); - kvm_x86_ops->hwapic_irr_update(vcpu, + kvm_x86_ops_apicv_post_state_restore(vcpu); + kvm_x86_ops_hwapic_irr_update(vcpu, apic_find_highest_irr(apic)); - kvm_x86_ops->hwapic_isr_update(vcpu, + kvm_x86_ops_hwapic_isr_update(vcpu, apic_find_highest_isr(apic)); } kvm_make_request(KVM_REQ_EVENT, vcpu); diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index a10af9c87f8a..affeba031937 100644 --- a/arch/x86/kvm/mmu.c +++ b/arch/x86/kvm/mmu.c @@ -283,7 +283,7 @@ static void kvm_flush_remote_tlbs_with_range(struct kvm *kvm, int ret = -ENOTSUPP; if (range && kvm_x86_ops->tlb_remote_flush_with_range) - ret = kvm_x86_ops->tlb_remote_flush_with_range(kvm, range); + ret = kvm_x86_ops_tlb_remote_flush_with_range(kvm, range); if (ret) kvm_flush_remote_tlbs(kvm); @@ -1265,7 +1265,7 @@ static int mapping_level(struct kvm_vcpu *vcpu, gfn_t large_gfn, if (host_level == PT_PAGE_TABLE_LEVEL) return host_level; - max_level = min(kvm_x86_ops->get_lpage_level(), host_level); + max_level = min(kvm_x86_ops_get_lpage_level(), host_level); for (level = PT_DIRECTORY_LEVEL; level <= max_level; ++level) if (__mmu_gfn_lpage_is_disallowed(large_gfn, level, slot)) @@ -1719,7 +1719,7 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm, gfn_t gfn_offset, unsigned long mask) { if (kvm_x86_ops->enable_log_dirty_pt_masked) - kvm_x86_ops->enable_log_dirty_pt_masked(kvm, slot, gfn_offset, + kvm_x86_ops_enable_log_dirty_pt_masked(kvm, slot, gfn_offset, mask); else kvm_mmu_write_protect_pt_masked(kvm, slot, gfn_offset, mask); @@ -1735,7 +1735,7 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm, int kvm_arch_write_log_dirty(struct kvm_vcpu *vcpu) { if (kvm_x86_ops->write_log_dirty) - return kvm_x86_ops->write_log_dirty(vcpu); + return kvm_x86_ops_write_log_dirty(vcpu); return 0; } @@ -2987,7 +2987,7 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep, if (level > PT_PAGE_TABLE_LEVEL) spte |= PT_PAGE_SIZE_MASK; if (tdp_enabled) - spte |= kvm_x86_ops->get_mt_mask(vcpu, gfn, + spte |= kvm_x86_ops_get_mt_mask(vcpu, gfn, kvm_is_mmio_pfn(pfn)); if (host_writable) @@ -4258,7 +4258,7 @@ static bool fast_cr3_switch(struct kvm_vcpu *vcpu, gpa_t new_cr3, kvm_make_request(KVM_REQ_LOAD_CR3, vcpu); if (!skip_tlb_flush) { kvm_make_request(KVM_REQ_MMU_SYNC, vcpu); - kvm_x86_ops->tlb_flush(vcpu, true); + kvm_x86_ops_tlb_flush(vcpu, true); } /* @@ -4853,7 +4853,7 @@ kvm_calc_tdp_mmu_root_page_role(struct kvm_vcpu *vcpu, bool base_only) union kvm_mmu_role role = kvm_calc_mmu_role_common(vcpu, base_only); role.base.ad_disabled = (shadow_accessed_mask == 0); - role.base.level = kvm_x86_ops->get_tdp_level(vcpu); + role.base.level = kvm_x86_ops_get_tdp_level(vcpu); role.base.direct = true; role.base.gpte_is_8_bytes = true; @@ -4875,7 +4875,7 @@ static void init_kvm_tdp_mmu(struct kvm_vcpu *vcpu) context->sync_page = nonpaging_sync_page; context->invlpg = nonpaging_invlpg; context->update_pte = nonpaging_update_pte; - context->shadow_root_level = kvm_x86_ops->get_tdp_level(vcpu); + context->shadow_root_level = kvm_x86_ops_get_tdp_level(vcpu); context->direct_map = true; context->set_cr3 = kvm_x86_ops->set_tdp_cr3; context->get_cr3 = get_cr3; @@ -5132,7 +5132,7 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu) if (r) goto out; kvm_mmu_load_cr3(vcpu); - kvm_x86_ops->tlb_flush(vcpu, true); + kvm_x86_ops_tlb_flush(vcpu, true); out: return r; } @@ -5446,7 +5446,7 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u64 error_code, * guest, with the exception of AMD Erratum 1096 which is unrecoverable. */ if (unlikely(insn && !insn_len)) { - if (!kvm_x86_ops->need_emulation_on_page_fault(vcpu)) + if (!kvm_x86_ops_need_emulation_on_page_fault(vcpu)) return 1; } @@ -5492,7 +5492,7 @@ void kvm_mmu_invlpg(struct kvm_vcpu *vcpu, gva_t gva) if (VALID_PAGE(mmu->prev_roots[i].hpa)) mmu->invlpg(vcpu, gva, mmu->prev_roots[i].hpa); - kvm_x86_ops->tlb_flush_gva(vcpu, gva); + kvm_x86_ops_tlb_flush_gva(vcpu, gva); ++vcpu->stat.invlpg; } EXPORT_SYMBOL_GPL(kvm_mmu_invlpg); @@ -5517,7 +5517,7 @@ void kvm_mmu_invpcid_gva(struct kvm_vcpu *vcpu, gva_t gva, unsigned long pcid) } if (tlb_flush) - kvm_x86_ops->tlb_flush_gva(vcpu, gva); + kvm_x86_ops_tlb_flush_gva(vcpu, gva); ++vcpu->stat.invlpg; @@ -5634,7 +5634,7 @@ static int alloc_mmu_pages(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu) * SVM's 32-bit NPT support, TDP paging doesn't use PAE paging and can * skip allocating the PDP table. */ - if (tdp_enabled && kvm_x86_ops->get_tdp_level(vcpu) > PT32E_ROOT_LEVEL) + if (tdp_enabled && kvm_x86_ops_get_tdp_level(vcpu) > PT32E_ROOT_LEVEL) return 0; page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_DMA32); diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 11f8ec89433b..c941980f58a4 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -157,8 +157,8 @@ static inline u8 permission_fault(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned pte_access, unsigned pte_pkey, unsigned pfec) { - int cpl = kvm_x86_ops->get_cpl(vcpu); - unsigned long rflags = kvm_x86_ops->get_rflags(vcpu); + int cpl = kvm_x86_ops_get_cpl(vcpu); + unsigned long rflags = kvm_x86_ops_get_rflags(vcpu); /* * If CPL < 3, SMAP prevention are disabled if EFLAGS.AC = 1. diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 46875bbd0419..363d3f86fb99 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -183,7 +183,7 @@ void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel) ARCH_PERFMON_EVENTSEL_CMASK | HSW_IN_TX | HSW_IN_TX_CHECKPOINTED))) { - config = kvm_x86_ops->pmu_ops->find_arch_event(pmc_to_pmu(pmc), + config = kvm_pmu_ops_find_arch_event(pmc_to_pmu(pmc), event_select, unit_mask); if (config != PERF_COUNT_HW_MAX) @@ -225,7 +225,7 @@ void reprogram_fixed_counter(struct kvm_pmc *pmc, u8 ctrl, int idx) } pmc_reprogram_counter(pmc, PERF_TYPE_HARDWARE, - kvm_x86_ops->pmu_ops->find_fixed_event(idx), + kvm_pmu_ops_find_fixed_event(idx), !(en_field & 0x2), /* exclude user */ !(en_field & 0x1), /* exclude kernel */ pmi, false, false); @@ -234,7 +234,7 @@ EXPORT_SYMBOL_GPL(reprogram_fixed_counter); void reprogram_counter(struct kvm_pmu *pmu, int pmc_idx) { - struct kvm_pmc *pmc = kvm_x86_ops->pmu_ops->pmc_idx_to_pmc(pmu, pmc_idx); + struct kvm_pmc *pmc = kvm_pmu_ops_pmc_idx_to_pmc(pmu, pmc_idx); if (!pmc) return; @@ -259,7 +259,7 @@ void kvm_pmu_handle_event(struct kvm_vcpu *vcpu) bitmask = pmu->reprogram_pmi; for_each_set_bit(bit, (unsigned long *)&bitmask, X86_PMC_IDX_MAX) { - struct kvm_pmc *pmc = kvm_x86_ops->pmu_ops->pmc_idx_to_pmc(pmu, bit); + struct kvm_pmc *pmc = kvm_pmu_ops_pmc_idx_to_pmc(pmu, bit); if (unlikely(!pmc || !pmc->perf_event)) { clear_bit(bit, (unsigned long *)&pmu->reprogram_pmi); @@ -273,7 +273,7 @@ void kvm_pmu_handle_event(struct kvm_vcpu *vcpu) /* check if idx is a valid index to access PMU */ int kvm_pmu_is_valid_msr_idx(struct kvm_vcpu *vcpu, unsigned idx) { - return kvm_x86_ops->pmu_ops->is_valid_msr_idx(vcpu, idx); + return kvm_pmu_ops_is_valid_msr_idx(vcpu, idx); } bool is_vmware_backdoor_pmc(u32 pmc_idx) @@ -323,7 +323,7 @@ int kvm_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned idx, u64 *data) if (is_vmware_backdoor_pmc(idx)) return kvm_pmu_rdpmc_vmware(vcpu, idx, data); - pmc = kvm_x86_ops->pmu_ops->msr_idx_to_pmc(vcpu, idx, &mask); + pmc = kvm_pmu_ops_msr_idx_to_pmc(vcpu, idx, &mask); if (!pmc) return 1; @@ -339,17 +339,17 @@ void kvm_pmu_deliver_pmi(struct kvm_vcpu *vcpu) bool kvm_pmu_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr) { - return kvm_x86_ops->pmu_ops->is_valid_msr(vcpu, msr); + return kvm_pmu_ops_is_valid_msr(vcpu, msr); } int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *data) { - return kvm_x86_ops->pmu_ops->get_msr(vcpu, msr, data); + return kvm_pmu_ops_get_msr(vcpu, msr, data); } int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) { - return kvm_x86_ops->pmu_ops->set_msr(vcpu, msr_info); + return kvm_pmu_ops_set_msr(vcpu, msr_info); } /* refresh PMU settings. This function generally is called when underlying @@ -358,7 +358,7 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) */ void kvm_pmu_refresh(struct kvm_vcpu *vcpu) { - kvm_x86_ops->pmu_ops->refresh(vcpu); + kvm_pmu_ops_refresh(vcpu); } void kvm_pmu_reset(struct kvm_vcpu *vcpu) @@ -366,7 +366,7 @@ void kvm_pmu_reset(struct kvm_vcpu *vcpu) struct kvm_pmu *pmu = vcpu_to_pmu(vcpu); irq_work_sync(&pmu->irq_work); - kvm_x86_ops->pmu_ops->reset(vcpu); + kvm_pmu_ops_reset(vcpu); } void kvm_pmu_init(struct kvm_vcpu *vcpu) @@ -374,7 +374,7 @@ void kvm_pmu_init(struct kvm_vcpu *vcpu) struct kvm_pmu *pmu = vcpu_to_pmu(vcpu); memset(pmu, 0, sizeof(*pmu)); - kvm_x86_ops->pmu_ops->init(vcpu); + kvm_pmu_ops_init(vcpu); init_irq_work(&pmu->irq_work, kvm_pmi_trigger_fn); kvm_pmu_refresh(vcpu); } diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index a9c2f68a40cc..09e80e8ee21a 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -78,7 +78,7 @@ static inline bool pmc_is_fixed(struct kvm_pmc *pmc) static inline bool pmc_is_enabled(struct kvm_pmc *pmc) { - return kvm_x86_ops->pmu_ops->pmc_is_enabled(pmc); + return kvm_pmu_ops_pmc_is_enabled(pmc); } /* returns general purpose PMC with the specified MSR. Note that it can be diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h index 7c741a0c5f80..f9994efc4c44 100644 --- a/arch/x86/kvm/trace.h +++ b/arch/x86/kvm/trace.h @@ -240,7 +240,7 @@ TRACE_EVENT(kvm_exit, __entry->guest_rip = kvm_rip_read(vcpu); __entry->isa = isa; __entry->vcpu_id = vcpu->vcpu_id; - kvm_x86_ops->get_exit_info(vcpu, &__entry->info1, + kvm_x86_ops_get_exit_info(vcpu, &__entry->info1, &__entry->info2); ), @@ -744,7 +744,7 @@ TRACE_EVENT(kvm_emulate_insn, ), TP_fast_assign( - __entry->csbase = kvm_x86_ops->get_segment_base(vcpu, VCPU_SREG_CS); + __entry->csbase = kvm_x86_ops_get_segment_base(vcpu, VCPU_SREG_CS); __entry->len = vcpu->arch.emulate_ctxt.fetch.ptr - vcpu->arch.emulate_ctxt.fetch.data; __entry->rip = vcpu->arch.emulate_ctxt._eip - __entry->len; diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 5ef36a75c31e..bfa765842772 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -305,7 +305,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu) pmu->global_ovf_ctrl_mask = pmu->global_ctrl_mask & ~(MSR_CORE_PERF_GLOBAL_OVF_CTRL_OVF_BUF | MSR_CORE_PERF_GLOBAL_OVF_CTRL_COND_CHGD); - if (kvm_x86_ops->pt_supported()) + if (kvm_x86_ops_pt_supported()) pmu->global_ovf_ctrl_mask &= ~MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c04894b61384..2db3c0cb2631 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -626,7 +626,7 @@ EXPORT_SYMBOL_GPL(kvm_requeue_exception_e); */ bool kvm_require_cpl(struct kvm_vcpu *vcpu, int required_cpl) { - if (kvm_x86_ops->get_cpl(vcpu) <= required_cpl) + if (kvm_x86_ops_get_cpl(vcpu) <= required_cpl) return true; kvm_queue_exception_e(vcpu, GP_VECTOR, 0); return false; @@ -773,7 +773,7 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) if (!is_pae(vcpu)) return 1; - kvm_x86_ops->get_cs_db_l_bits(vcpu, &cs_db, &cs_l); + kvm_x86_ops_get_cs_db_l_bits(vcpu, &cs_db, &cs_l); if (cs_l) return 1; } else @@ -786,7 +786,7 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) if (!(cr0 & X86_CR0_PG) && kvm_read_cr4_bits(vcpu, X86_CR4_PCIDE)) return 1; - kvm_x86_ops->set_cr0(vcpu, cr0); + kvm_x86_ops_set_cr0(vcpu, cr0); if ((cr0 ^ old_cr0) & X86_CR0_PG) { kvm_clear_async_pf_completion_queue(vcpu); @@ -875,7 +875,7 @@ static int __kvm_set_xcr(struct kvm_vcpu *vcpu, u32 index, u64 xcr) int kvm_set_xcr(struct kvm_vcpu *vcpu, u32 index, u64 xcr) { - if (kvm_x86_ops->get_cpl(vcpu) != 0 || + if (kvm_x86_ops_get_cpl(vcpu) != 0 || __kvm_set_xcr(vcpu, index, xcr)) { kvm_inject_gp(vcpu, 0); return 1; @@ -932,7 +932,7 @@ int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) return 1; } - if (kvm_x86_ops->set_cr4(vcpu, cr4)) + if (kvm_x86_ops_set_cr4(vcpu, cr4)) return 1; if (((cr4 ^ old_cr4) & pdptr_bits) || @@ -1016,7 +1016,7 @@ static void kvm_update_dr0123(struct kvm_vcpu *vcpu) static void kvm_update_dr6(struct kvm_vcpu *vcpu) { if (!(vcpu->guest_debug & KVM_GUESTDBG_USE_HW_BP)) - kvm_x86_ops->set_dr6(vcpu, vcpu->arch.dr6); + kvm_x86_ops_set_dr6(vcpu, vcpu->arch.dr6); } static void kvm_update_dr7(struct kvm_vcpu *vcpu) @@ -1027,7 +1027,7 @@ static void kvm_update_dr7(struct kvm_vcpu *vcpu) dr7 = vcpu->arch.guest_debug_dr7; else dr7 = vcpu->arch.dr7; - kvm_x86_ops->set_dr7(vcpu, dr7); + kvm_x86_ops_set_dr7(vcpu, dr7); vcpu->arch.switch_db_regs &= ~KVM_DEBUGREG_BP_ENABLED; if (dr7 & DR7_BP_EN_MASK) vcpu->arch.switch_db_regs |= KVM_DEBUGREG_BP_ENABLED; @@ -1093,7 +1093,7 @@ int kvm_get_dr(struct kvm_vcpu *vcpu, int dr, unsigned long *val) if (vcpu->guest_debug & KVM_GUESTDBG_USE_HW_BP) *val = vcpu->arch.dr6; else - *val = kvm_x86_ops->get_dr6(vcpu); + *val = kvm_x86_ops_get_dr6(vcpu); break; case 5: /* fall through */ @@ -1279,7 +1279,7 @@ static int kvm_get_msr_feature(struct kvm_msr_entry *msr) rdmsrl_safe(msr->index, &msr->data); break; default: - if (kvm_x86_ops->get_msr_feature(msr)) + if (kvm_x86_ops_get_msr_feature(msr)) return 1; } return 0; @@ -1347,7 +1347,7 @@ static int set_efer(struct kvm_vcpu *vcpu, struct msr_data *msr_info) efer &= ~EFER_LMA; efer |= vcpu->arch.efer & EFER_LMA; - kvm_x86_ops->set_efer(vcpu, efer); + kvm_x86_ops_set_efer(vcpu, efer); /* Update reserved bits */ if ((efer ^ old_efer) & EFER_NX) @@ -1403,7 +1403,7 @@ static int __kvm_set_msr(struct kvm_vcpu *vcpu, u32 index, u64 data, msr.index = index; msr.host_initiated = host_initiated; - return kvm_x86_ops->set_msr(vcpu, &msr); + return kvm_x86_ops_set_msr(vcpu, &msr); } /* @@ -1742,7 +1742,7 @@ static void kvm_track_tsc_matching(struct kvm_vcpu *vcpu) static void update_ia32_tsc_adjust_msr(struct kvm_vcpu *vcpu, s64 offset) { - u64 curr_offset = kvm_x86_ops->read_l1_tsc_offset(vcpu); + u64 curr_offset = kvm_x86_ops_read_l1_tsc_offset(vcpu); vcpu->arch.ia32_tsc_adjust_msr += offset - curr_offset; } @@ -1784,7 +1784,7 @@ static u64 kvm_compute_tsc_offset(struct kvm_vcpu *vcpu, u64 target_tsc) u64 kvm_read_l1_tsc(struct kvm_vcpu *vcpu, u64 host_tsc) { - u64 tsc_offset = kvm_x86_ops->read_l1_tsc_offset(vcpu); + u64 tsc_offset = kvm_x86_ops_read_l1_tsc_offset(vcpu); return tsc_offset + kvm_scale_tsc(vcpu, host_tsc); } @@ -1792,7 +1792,7 @@ EXPORT_SYMBOL_GPL(kvm_read_l1_tsc); static void kvm_vcpu_write_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) { - vcpu->arch.tsc_offset = kvm_x86_ops->write_l1_tsc_offset(vcpu, offset); + vcpu->arch.tsc_offset = kvm_x86_ops_write_l1_tsc_offset(vcpu, offset); } static inline bool kvm_check_tsc_unstable(void) @@ -1916,7 +1916,7 @@ EXPORT_SYMBOL_GPL(kvm_write_tsc); static inline void adjust_tsc_offset_guest(struct kvm_vcpu *vcpu, s64 adjustment) { - u64 tsc_offset = kvm_x86_ops->read_l1_tsc_offset(vcpu); + u64 tsc_offset = kvm_x86_ops_read_l1_tsc_offset(vcpu); kvm_vcpu_write_tsc_offset(vcpu, tsc_offset + adjustment); } @@ -2509,7 +2509,7 @@ static void kvmclock_reset(struct kvm_vcpu *vcpu) static void kvm_vcpu_flush_tlb(struct kvm_vcpu *vcpu, bool invalidate_gpa) { ++vcpu->stat.tlb_flush; - kvm_x86_ops->tlb_flush(vcpu, invalidate_gpa); + kvm_x86_ops_tlb_flush(vcpu, invalidate_gpa); } static void record_steal_time(struct kvm_vcpu *vcpu) @@ -3213,10 +3213,10 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) * fringe case that is not enabled except via specific settings * of the module parameters. */ - r = kvm_x86_ops->has_emulated_msr(MSR_IA32_SMBASE); + r = kvm_x86_ops_has_emulated_msr(MSR_IA32_SMBASE); break; case KVM_CAP_VAPIC: - r = !kvm_x86_ops->cpu_has_accelerated_tpr(); + r = !kvm_x86_ops_cpu_has_accelerated_tpr(); break; case KVM_CAP_NR_VCPUS: r = KVM_SOFT_MAX_VCPUS; @@ -3244,7 +3244,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) break; case KVM_CAP_NESTED_STATE: r = kvm_x86_ops->get_nested_state ? - kvm_x86_ops->get_nested_state(NULL, NULL, 0) : 0; + kvm_x86_ops_get_nested_state(NULL, NULL, 0) : 0; break; default: break; @@ -3360,14 +3360,14 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) { /* Address WBINVD may be executed by guest */ if (need_emulate_wbinvd(vcpu)) { - if (kvm_x86_ops->has_wbinvd_exit()) + if (kvm_x86_ops_has_wbinvd_exit()) cpumask_set_cpu(cpu, vcpu->arch.wbinvd_dirty_mask); else if (vcpu->cpu != -1 && vcpu->cpu != cpu) smp_call_function_single(vcpu->cpu, wbinvd_ipi, NULL, 1); } - kvm_x86_ops->vcpu_load(vcpu, cpu); + kvm_x86_ops_vcpu_load(vcpu, cpu); fpregs_assert_state_consistent(); if (test_thread_flag(TIF_NEED_FPU_LOAD)) @@ -3428,7 +3428,7 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) int idx; if (vcpu->preempted) - vcpu->arch.preempted_in_kernel = !kvm_x86_ops->get_cpl(vcpu); + vcpu->arch.preempted_in_kernel = !kvm_x86_ops_get_cpl(vcpu); /* * Disable page faults because we're in atomic context here. @@ -3447,7 +3447,7 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) kvm_steal_time_set_preempted(vcpu); srcu_read_unlock(&vcpu->kvm->srcu, idx); pagefault_enable(); - kvm_x86_ops->vcpu_put(vcpu); + kvm_x86_ops_vcpu_put(vcpu); vcpu->arch.last_host_tsc = rdtsc(); /* * If userspace has set any breakpoints or watchpoints, dr6 is restored @@ -3461,7 +3461,7 @@ static int kvm_vcpu_ioctl_get_lapic(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s) { if (vcpu->arch.apicv_active) - kvm_x86_ops->sync_pir_to_irr(vcpu); + kvm_x86_ops_sync_pir_to_irr(vcpu); return kvm_apic_get_state(vcpu, s); } @@ -3569,7 +3569,7 @@ static int kvm_vcpu_ioctl_x86_setup_mce(struct kvm_vcpu *vcpu, for (bank = 0; bank < bank_num; bank++) vcpu->arch.mce_banks[bank*4] = ~(u64)0; - kvm_x86_ops->setup_mce(vcpu); + kvm_x86_ops_setup_mce(vcpu); out: return r; } @@ -3658,11 +3658,11 @@ static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu *vcpu, vcpu->arch.interrupt.injected && !vcpu->arch.interrupt.soft; events->interrupt.nr = vcpu->arch.interrupt.nr; events->interrupt.soft = 0; - events->interrupt.shadow = kvm_x86_ops->get_interrupt_shadow(vcpu); + events->interrupt.shadow = kvm_x86_ops_get_interrupt_shadow(vcpu); events->nmi.injected = vcpu->arch.nmi_injected; events->nmi.pending = vcpu->arch.nmi_pending != 0; - events->nmi.masked = kvm_x86_ops->get_nmi_mask(vcpu); + events->nmi.masked = kvm_x86_ops_get_nmi_mask(vcpu); events->nmi.pad = 0; events->sipi_vector = 0; /* never valid when reporting to user space */ @@ -3729,13 +3729,13 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu *vcpu, vcpu->arch.interrupt.nr = events->interrupt.nr; vcpu->arch.interrupt.soft = events->interrupt.soft; if (events->flags & KVM_VCPUEVENT_VALID_SHADOW) - kvm_x86_ops->set_interrupt_shadow(vcpu, + kvm_x86_ops_set_interrupt_shadow(vcpu, events->interrupt.shadow); vcpu->arch.nmi_injected = events->nmi.injected; if (events->flags & KVM_VCPUEVENT_VALID_NMI_PENDING) vcpu->arch.nmi_pending = events->nmi.pending; - kvm_x86_ops->set_nmi_mask(vcpu, events->nmi.masked); + kvm_x86_ops_set_nmi_mask(vcpu, events->nmi.masked); if (events->flags & KVM_VCPUEVENT_VALID_SIPI_VECTOR && lapic_in_kernel(vcpu)) @@ -4011,7 +4011,7 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu, case KVM_CAP_HYPERV_ENLIGHTENED_VMCS: if (!kvm_x86_ops->nested_enable_evmcs) return -ENOTTY; - r = kvm_x86_ops->nested_enable_evmcs(vcpu, &vmcs_version); + r = kvm_x86_ops_nested_enable_evmcs(vcpu, &vmcs_version); if (!r) { user_ptr = (void __user *)(uintptr_t)cap->args[0]; if (copy_to_user(user_ptr, &vmcs_version, @@ -4329,7 +4329,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp, if (get_user(user_data_size, &user_kvm_nested_state->size)) break; - r = kvm_x86_ops->get_nested_state(vcpu, user_kvm_nested_state, + r = kvm_x86_ops_get_nested_state(vcpu, user_kvm_nested_state, user_data_size); if (r < 0) break; @@ -4371,7 +4371,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp, && !(kvm_state.flags & KVM_STATE_NESTED_GUEST_MODE)) break; - r = kvm_x86_ops->set_nested_state(vcpu, user_kvm_nested_state, &kvm_state); + r = kvm_x86_ops_set_nested_state(vcpu, user_kvm_nested_state, &kvm_state); break; } case KVM_GET_SUPPORTED_HV_CPUID: { @@ -4414,14 +4414,14 @@ static int kvm_vm_ioctl_set_tss_addr(struct kvm *kvm, unsigned long addr) if (addr > (unsigned int)(-3 * PAGE_SIZE)) return -EINVAL; - ret = kvm_x86_ops->set_tss_addr(kvm, addr); + ret = kvm_x86_ops_set_tss_addr(kvm, addr); return ret; } static int kvm_vm_ioctl_set_identity_map_addr(struct kvm *kvm, u64 ident_addr) { - return kvm_x86_ops->set_identity_map_addr(kvm, ident_addr); + return kvm_x86_ops_set_identity_map_addr(kvm, ident_addr); } static int kvm_vm_ioctl_set_nr_mmu_pages(struct kvm *kvm, @@ -4606,7 +4606,7 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log) * Flush potentially hardware-cached dirty pages to dirty_bitmap. */ if (kvm_x86_ops->flush_log_dirty) - kvm_x86_ops->flush_log_dirty(kvm); + kvm_x86_ops_flush_log_dirty(kvm); r = kvm_get_dirty_log_protect(kvm, log, &flush); @@ -4633,7 +4633,7 @@ int kvm_vm_ioctl_clear_dirty_log(struct kvm *kvm, struct kvm_clear_dirty_log *lo * Flush potentially hardware-cached dirty pages to dirty_bitmap. */ if (kvm_x86_ops->flush_log_dirty) - kvm_x86_ops->flush_log_dirty(kvm); + kvm_x86_ops_flush_log_dirty(kvm); r = kvm_clear_dirty_log_protect(kvm, log, &flush); @@ -5000,7 +5000,7 @@ long kvm_arch_vm_ioctl(struct file *filp, case KVM_MEMORY_ENCRYPT_OP: { r = -ENOTTY; if (kvm_x86_ops->mem_enc_op) - r = kvm_x86_ops->mem_enc_op(kvm, argp); + r = kvm_x86_ops_mem_enc_op(kvm, argp); break; } case KVM_MEMORY_ENCRYPT_REG_REGION: { @@ -5012,7 +5012,7 @@ long kvm_arch_vm_ioctl(struct file *filp, r = -ENOTTY; if (kvm_x86_ops->mem_enc_reg_region) - r = kvm_x86_ops->mem_enc_reg_region(kvm, ®ion); + r = kvm_x86_ops_mem_enc_reg_region(kvm, ®ion); break; } case KVM_MEMORY_ENCRYPT_UNREG_REGION: { @@ -5024,7 +5024,7 @@ long kvm_arch_vm_ioctl(struct file *filp, r = -ENOTTY; if (kvm_x86_ops->mem_enc_unreg_region) - r = kvm_x86_ops->mem_enc_unreg_region(kvm, ®ion); + r = kvm_x86_ops_mem_enc_unreg_region(kvm, ®ion); break; } case KVM_HYPERV_EVENTFD: { @@ -5065,28 +5065,28 @@ static void kvm_init_msr_list(void) continue; break; case MSR_TSC_AUX: - if (!kvm_x86_ops->rdtscp_supported()) + if (!kvm_x86_ops_rdtscp_supported()) continue; break; case MSR_IA32_RTIT_CTL: case MSR_IA32_RTIT_STATUS: - if (!kvm_x86_ops->pt_supported()) + if (!kvm_x86_ops_pt_supported()) continue; break; case MSR_IA32_RTIT_CR3_MATCH: - if (!kvm_x86_ops->pt_supported() || + if (!kvm_x86_ops_pt_supported() || !intel_pt_validate_hw_cap(PT_CAP_cr3_filtering)) continue; break; case MSR_IA32_RTIT_OUTPUT_BASE: case MSR_IA32_RTIT_OUTPUT_MASK: - if (!kvm_x86_ops->pt_supported() || + if (!kvm_x86_ops_pt_supported() || (!intel_pt_validate_hw_cap(PT_CAP_topa_output) && !intel_pt_validate_hw_cap(PT_CAP_single_range_output))) continue; break; case MSR_IA32_RTIT_ADDR0_A ... MSR_IA32_RTIT_ADDR3_B: { - if (!kvm_x86_ops->pt_supported() || + if (!kvm_x86_ops_pt_supported() || msrs_to_save[i] - MSR_IA32_RTIT_ADDR0_A >= intel_pt_validate_hw_cap(PT_CAP_num_address_ranges) * 2) continue; @@ -5103,7 +5103,7 @@ static void kvm_init_msr_list(void) num_msrs_to_save = j; for (i = j = 0; i < ARRAY_SIZE(emulated_msrs); i++) { - if (!kvm_x86_ops->has_emulated_msr(emulated_msrs[i])) + if (!kvm_x86_ops_has_emulated_msr(emulated_msrs[i])) continue; if (j < i) @@ -5172,13 +5172,13 @@ static int vcpu_mmio_read(struct kvm_vcpu *vcpu, gpa_t addr, int len, void *v) static void kvm_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg) { - kvm_x86_ops->set_segment(vcpu, var, seg); + kvm_x86_ops_set_segment(vcpu, var, seg); } void kvm_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg) { - kvm_x86_ops->get_segment(vcpu, var, seg); + kvm_x86_ops_get_segment(vcpu, var, seg); } gpa_t translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, u32 access, @@ -5198,14 +5198,14 @@ gpa_t translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, u32 access, gpa_t kvm_mmu_gva_to_gpa_read(struct kvm_vcpu *vcpu, gva_t gva, struct x86_exception *exception) { - u32 access = (kvm_x86_ops->get_cpl(vcpu) == 3) ? PFERR_USER_MASK : 0; + u32 access = (kvm_x86_ops_get_cpl(vcpu) == 3) ? PFERR_USER_MASK : 0; return vcpu->arch.walk_mmu->gva_to_gpa(vcpu, gva, access, exception); } gpa_t kvm_mmu_gva_to_gpa_fetch(struct kvm_vcpu *vcpu, gva_t gva, struct x86_exception *exception) { - u32 access = (kvm_x86_ops->get_cpl(vcpu) == 3) ? PFERR_USER_MASK : 0; + u32 access = (kvm_x86_ops_get_cpl(vcpu) == 3) ? PFERR_USER_MASK : 0; access |= PFERR_FETCH_MASK; return vcpu->arch.walk_mmu->gva_to_gpa(vcpu, gva, access, exception); } @@ -5213,7 +5213,7 @@ gpa_t kvm_mmu_gva_to_gpa_read(struct kvm_vcpu *vcpu, gva_t gva, gpa_t kvm_mmu_gva_to_gpa_write(struct kvm_vcpu *vcpu, gva_t gva, struct x86_exception *exception) { - u32 access = (kvm_x86_ops->get_cpl(vcpu) == 3) ? PFERR_USER_MASK : 0; + u32 access = (kvm_x86_ops_get_cpl(vcpu) == 3) ? PFERR_USER_MASK : 0; access |= PFERR_WRITE_MASK; return vcpu->arch.walk_mmu->gva_to_gpa(vcpu, gva, access, exception); } @@ -5262,7 +5262,7 @@ static int kvm_fetch_guest_virt(struct x86_emulate_ctxt *ctxt, struct x86_exception *exception) { struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt); - u32 access = (kvm_x86_ops->get_cpl(vcpu) == 3) ? PFERR_USER_MASK : 0; + u32 access = (kvm_x86_ops_get_cpl(vcpu) == 3) ? PFERR_USER_MASK : 0; unsigned offset; int ret; @@ -5287,7 +5287,7 @@ int kvm_read_guest_virt(struct kvm_vcpu *vcpu, gva_t addr, void *val, unsigned int bytes, struct x86_exception *exception) { - u32 access = (kvm_x86_ops->get_cpl(vcpu) == 3) ? PFERR_USER_MASK : 0; + u32 access = (kvm_x86_ops_get_cpl(vcpu) == 3) ? PFERR_USER_MASK : 0; /* * FIXME: this should call handle_emulation_failure if X86EMUL_IO_NEEDED @@ -5308,7 +5308,7 @@ static int emulator_read_std(struct x86_emulate_ctxt *ctxt, struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt); u32 access = 0; - if (!system && kvm_x86_ops->get_cpl(vcpu) == 3) + if (!system && kvm_x86_ops_get_cpl(vcpu) == 3) access |= PFERR_USER_MASK; return kvm_read_guest_virt_helper(addr, val, bytes, vcpu, access, exception); @@ -5361,7 +5361,7 @@ static int emulator_write_std(struct x86_emulate_ctxt *ctxt, gva_t addr, void *v struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt); u32 access = PFERR_WRITE_MASK; - if (!system && kvm_x86_ops->get_cpl(vcpu) == 3) + if (!system && kvm_x86_ops_get_cpl(vcpu) == 3) access |= PFERR_USER_MASK; return kvm_write_guest_virt_helper(addr, val, bytes, vcpu, @@ -5429,7 +5429,7 @@ static int vcpu_mmio_gva_to_gpa(struct kvm_vcpu *vcpu, unsigned long gva, gpa_t *gpa, struct x86_exception *exception, bool write) { - u32 access = ((kvm_x86_ops->get_cpl(vcpu) == 3) ? PFERR_USER_MASK : 0) + u32 access = ((kvm_x86_ops_get_cpl(vcpu) == 3) ? PFERR_USER_MASK : 0) | (write ? PFERR_WRITE_MASK : 0); /* @@ -5817,7 +5817,7 @@ static int emulator_pio_out_emulated(struct x86_emulate_ctxt *ctxt, static unsigned long get_segment_base(struct kvm_vcpu *vcpu, int seg) { - return kvm_x86_ops->get_segment_base(vcpu, seg); + return kvm_x86_ops_get_segment_base(vcpu, seg); } static void emulator_invlpg(struct x86_emulate_ctxt *ctxt, ulong address) @@ -5830,7 +5830,7 @@ static int kvm_emulate_wbinvd_noskip(struct kvm_vcpu *vcpu) if (!need_emulate_wbinvd(vcpu)) return X86EMUL_CONTINUE; - if (kvm_x86_ops->has_wbinvd_exit()) { + if (kvm_x86_ops_has_wbinvd_exit()) { int cpu = get_cpu(); cpumask_set_cpu(cpu, vcpu->arch.wbinvd_dirty_mask); @@ -5935,27 +5935,27 @@ static int emulator_set_cr(struct x86_emulate_ctxt *ctxt, int cr, ulong val) static int emulator_get_cpl(struct x86_emulate_ctxt *ctxt) { - return kvm_x86_ops->get_cpl(emul_to_vcpu(ctxt)); + return kvm_x86_ops_get_cpl(emul_to_vcpu(ctxt)); } static void emulator_get_gdt(struct x86_emulate_ctxt *ctxt, struct desc_ptr *dt) { - kvm_x86_ops->get_gdt(emul_to_vcpu(ctxt), dt); + kvm_x86_ops_get_gdt(emul_to_vcpu(ctxt), dt); } static void emulator_get_idt(struct x86_emulate_ctxt *ctxt, struct desc_ptr *dt) { - kvm_x86_ops->get_idt(emul_to_vcpu(ctxt), dt); + kvm_x86_ops_get_idt(emul_to_vcpu(ctxt), dt); } static void emulator_set_gdt(struct x86_emulate_ctxt *ctxt, struct desc_ptr *dt) { - kvm_x86_ops->set_gdt(emul_to_vcpu(ctxt), dt); + kvm_x86_ops_set_gdt(emul_to_vcpu(ctxt), dt); } static void emulator_set_idt(struct x86_emulate_ctxt *ctxt, struct desc_ptr *dt) { - kvm_x86_ops->set_idt(emul_to_vcpu(ctxt), dt); + kvm_x86_ops_set_idt(emul_to_vcpu(ctxt), dt); } static unsigned long emulator_get_cached_segment_base( @@ -6077,7 +6077,7 @@ static int emulator_intercept(struct x86_emulate_ctxt *ctxt, struct x86_instruction_info *info, enum x86_intercept_stage stage) { - return kvm_x86_ops->check_intercept(emul_to_vcpu(ctxt), info, stage); + return kvm_x86_ops_check_intercept(emul_to_vcpu(ctxt), info, stage); } static bool emulator_get_cpuid(struct x86_emulate_ctxt *ctxt, @@ -6098,7 +6098,7 @@ static void emulator_write_gpr(struct x86_emulate_ctxt *ctxt, unsigned reg, ulon static void emulator_set_nmi_mask(struct x86_emulate_ctxt *ctxt, bool masked) { - kvm_x86_ops->set_nmi_mask(emul_to_vcpu(ctxt), masked); + kvm_x86_ops_set_nmi_mask(emul_to_vcpu(ctxt), masked); } static unsigned emulator_get_hflags(struct x86_emulate_ctxt *ctxt) @@ -6114,7 +6114,7 @@ static void emulator_set_hflags(struct x86_emulate_ctxt *ctxt, unsigned emul_fla static int emulator_pre_leave_smm(struct x86_emulate_ctxt *ctxt, const char *smstate) { - return kvm_x86_ops->pre_leave_smm(emul_to_vcpu(ctxt), smstate); + return kvm_x86_ops_pre_leave_smm(emul_to_vcpu(ctxt), smstate); } static void emulator_post_leave_smm(struct x86_emulate_ctxt *ctxt) @@ -6173,7 +6173,7 @@ static const struct x86_emulate_ops emulate_ops = { static void toggle_interruptibility(struct kvm_vcpu *vcpu, u32 mask) { - u32 int_shadow = kvm_x86_ops->get_interrupt_shadow(vcpu); + u32 int_shadow = kvm_x86_ops_get_interrupt_shadow(vcpu); /* * an sti; sti; sequence only disable interrupts for the first * instruction. So, if the last instruction, be it emulated or @@ -6184,7 +6184,7 @@ static void toggle_interruptibility(struct kvm_vcpu *vcpu, u32 mask) if (int_shadow & mask) mask = 0; if (unlikely(int_shadow || mask)) { - kvm_x86_ops->set_interrupt_shadow(vcpu, mask); + kvm_x86_ops_set_interrupt_shadow(vcpu, mask); if (!mask) kvm_make_request(KVM_REQ_EVENT, vcpu); } @@ -6209,7 +6209,7 @@ static void init_emulate_ctxt(struct kvm_vcpu *vcpu) struct x86_emulate_ctxt *ctxt = &vcpu->arch.emulate_ctxt; int cs_db, cs_l; - kvm_x86_ops->get_cs_db_l_bits(vcpu, &cs_db, &cs_l); + kvm_x86_ops_get_cs_db_l_bits(vcpu, &cs_db, &cs_l); ctxt->eflags = kvm_get_rflags(vcpu); ctxt->tf = (ctxt->eflags & X86_EFLAGS_TF) != 0; @@ -6261,7 +6261,7 @@ static int handle_emulation_failure(struct kvm_vcpu *vcpu, int emulation_type) if (emulation_type & EMULTYPE_NO_UD_ON_FAIL) return EMULATE_FAIL; - if (!is_guest_mode(vcpu) && kvm_x86_ops->get_cpl(vcpu) == 0) { + if (!is_guest_mode(vcpu) && kvm_x86_ops_get_cpl(vcpu) == 0) { vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR; vcpu->run->internal.suberror = KVM_INTERNAL_ERROR_EMULATION; vcpu->run->internal.ndata = 0; @@ -6442,10 +6442,10 @@ static void kvm_vcpu_do_singlestep(struct kvm_vcpu *vcpu, int *r) int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu) { - unsigned long rflags = kvm_x86_ops->get_rflags(vcpu); + unsigned long rflags = kvm_x86_ops_get_rflags(vcpu); int r; - r = kvm_x86_ops->skip_emulated_instruction(vcpu); + r = kvm_x86_ops_skip_emulated_instruction(vcpu); if (unlikely(r != EMULATE_DONE)) return 0; @@ -6662,7 +6662,7 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, r = EMULATE_DONE; if (writeback) { - unsigned long rflags = kvm_x86_ops->get_rflags(vcpu); + unsigned long rflags = kvm_x86_ops_get_rflags(vcpu); toggle_interruptibility(vcpu, ctxt->interruptibility); vcpu->arch.emulate_regs_need_sync_to_vcpu = false; if (!ctxt->have_exception || @@ -6998,7 +6998,7 @@ static int kvm_is_user_mode(void) int user_mode = 3; if (__this_cpu_read(current_vcpu)) - user_mode = kvm_x86_ops->get_cpl(__this_cpu_read(current_vcpu)); + user_mode = kvm_x86_ops_get_cpl(__this_cpu_read(current_vcpu)); return user_mode != 0; } @@ -7262,7 +7262,7 @@ void kvm_vcpu_deactivate_apicv(struct kvm_vcpu *vcpu) return; vcpu->arch.apicv_active = false; - kvm_x86_ops->refresh_apicv_exec_ctrl(vcpu); + kvm_x86_ops_refresh_apicv_exec_ctrl(vcpu); } static void kvm_sched_yield(struct kvm *kvm, unsigned long dest_id) @@ -7307,7 +7307,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) a3 &= 0xFFFFFFFF; } - if (kvm_x86_ops->get_cpl(vcpu) != 0) { + if (kvm_x86_ops_get_cpl(vcpu) != 0) { ret = -KVM_EPERM; goto out; } @@ -7353,7 +7353,7 @@ static int emulator_fix_hypercall(struct x86_emulate_ctxt *ctxt) char instruction[3]; unsigned long rip = kvm_rip_read(vcpu); - kvm_x86_ops->patch_hypercall(vcpu, instruction); + kvm_x86_ops_patch_hypercall(vcpu, instruction); return emulator_write_emulated(ctxt, rip, instruction, 3, &ctxt->exception); @@ -7401,7 +7401,7 @@ static void update_cr8_intercept(struct kvm_vcpu *vcpu) tpr = kvm_lapic_get_cr8(vcpu); - kvm_x86_ops->update_cr8_intercept(vcpu, tpr, max_irr); + kvm_x86_ops_update_cr8_intercept(vcpu, tpr, max_irr); } static int inject_pending_event(struct kvm_vcpu *vcpu, bool req_int_win) @@ -7411,7 +7411,7 @@ static int inject_pending_event(struct kvm_vcpu *vcpu, bool req_int_win) /* try to reinject previous events if any */ if (vcpu->arch.exception.injected) - kvm_x86_ops->queue_exception(vcpu); + kvm_x86_ops_queue_exception(vcpu); /* * Do not inject an NMI or interrupt if there is a pending * exception. Exceptions and interrupts are recognized at @@ -7428,9 +7428,9 @@ static int inject_pending_event(struct kvm_vcpu *vcpu, bool req_int_win) */ else if (!vcpu->arch.exception.pending) { if (vcpu->arch.nmi_injected) - kvm_x86_ops->set_nmi(vcpu); + kvm_x86_ops_set_nmi(vcpu); else if (vcpu->arch.interrupt.injected) - kvm_x86_ops->set_irq(vcpu); + kvm_x86_ops_set_irq(vcpu); } /* @@ -7440,7 +7440,7 @@ static int inject_pending_event(struct kvm_vcpu *vcpu, bool req_int_win) * from L2 to L1. */ if (is_guest_mode(vcpu) && kvm_x86_ops->check_nested_events) { - r = kvm_x86_ops->check_nested_events(vcpu, req_int_win); + r = kvm_x86_ops_check_nested_events(vcpu, req_int_win); if (r != 0) return r; } @@ -7477,7 +7477,7 @@ static int inject_pending_event(struct kvm_vcpu *vcpu, bool req_int_win) } } - kvm_x86_ops->queue_exception(vcpu); + kvm_x86_ops_queue_exception(vcpu); } /* Don't consider new event if we re-injected an event */ @@ -7485,14 +7485,14 @@ static int inject_pending_event(struct kvm_vcpu *vcpu, bool req_int_win) return 0; if (vcpu->arch.smi_pending && !is_smm(vcpu) && - kvm_x86_ops->smi_allowed(vcpu)) { + kvm_x86_ops_smi_allowed(vcpu)) { vcpu->arch.smi_pending = false; ++vcpu->arch.smi_count; enter_smm(vcpu); - } else if (vcpu->arch.nmi_pending && kvm_x86_ops->nmi_allowed(vcpu)) { + } else if (vcpu->arch.nmi_pending && kvm_x86_ops_nmi_allowed(vcpu)) { --vcpu->arch.nmi_pending; vcpu->arch.nmi_injected = true; - kvm_x86_ops->set_nmi(vcpu); + kvm_x86_ops_set_nmi(vcpu); } else if (kvm_cpu_has_injectable_intr(vcpu)) { /* * Because interrupts can be injected asynchronously, we are @@ -7502,14 +7502,14 @@ static int inject_pending_event(struct kvm_vcpu *vcpu, bool req_int_win) * KVM_REQ_EVENT only on certain events and not unconditionally? */ if (is_guest_mode(vcpu) && kvm_x86_ops->check_nested_events) { - r = kvm_x86_ops->check_nested_events(vcpu, req_int_win); + r = kvm_x86_ops_check_nested_events(vcpu, req_int_win); if (r != 0) return r; } - if (kvm_x86_ops->interrupt_allowed(vcpu)) { + if (kvm_x86_ops_interrupt_allowed(vcpu)) { kvm_queue_interrupt(vcpu, kvm_cpu_get_interrupt(vcpu), false); - kvm_x86_ops->set_irq(vcpu); + kvm_x86_ops_set_irq(vcpu); } } @@ -7525,7 +7525,7 @@ static void process_nmi(struct kvm_vcpu *vcpu) * If an NMI is already in progress, limit further NMIs to just one. * Otherwise, allow two (and we'll inject the first one immediately). */ - if (kvm_x86_ops->get_nmi_mask(vcpu) || vcpu->arch.nmi_injected) + if (kvm_x86_ops_get_nmi_mask(vcpu) || vcpu->arch.nmi_injected) limit = 1; vcpu->arch.nmi_pending += atomic_xchg(&vcpu->arch.nmi_queued, 0); @@ -7615,11 +7615,11 @@ static void enter_smm_save_state_32(struct kvm_vcpu *vcpu, char *buf) put_smstate(u32, buf, 0x7f7c, seg.limit); put_smstate(u32, buf, 0x7f78, enter_smm_get_segment_flags(&seg)); - kvm_x86_ops->get_gdt(vcpu, &dt); + kvm_x86_ops_get_gdt(vcpu, &dt); put_smstate(u32, buf, 0x7f74, dt.address); put_smstate(u32, buf, 0x7f70, dt.size); - kvm_x86_ops->get_idt(vcpu, &dt); + kvm_x86_ops_get_idt(vcpu, &dt); put_smstate(u32, buf, 0x7f58, dt.address); put_smstate(u32, buf, 0x7f54, dt.size); @@ -7669,7 +7669,7 @@ static void enter_smm_save_state_64(struct kvm_vcpu *vcpu, char *buf) put_smstate(u32, buf, 0x7e94, seg.limit); put_smstate(u64, buf, 0x7e98, seg.base); - kvm_x86_ops->get_idt(vcpu, &dt); + kvm_x86_ops_get_idt(vcpu, &dt); put_smstate(u32, buf, 0x7e84, dt.size); put_smstate(u64, buf, 0x7e88, dt.address); @@ -7679,7 +7679,7 @@ static void enter_smm_save_state_64(struct kvm_vcpu *vcpu, char *buf) put_smstate(u32, buf, 0x7e74, seg.limit); put_smstate(u64, buf, 0x7e78, seg.base); - kvm_x86_ops->get_gdt(vcpu, &dt); + kvm_x86_ops_get_gdt(vcpu, &dt); put_smstate(u32, buf, 0x7e64, dt.size); put_smstate(u64, buf, 0x7e68, dt.address); @@ -7709,28 +7709,28 @@ static void enter_smm(struct kvm_vcpu *vcpu) * vCPU state (e.g. leave guest mode) after we've saved the state into * the SMM state-save area. */ - kvm_x86_ops->pre_enter_smm(vcpu, buf); + kvm_x86_ops_pre_enter_smm(vcpu, buf); vcpu->arch.hflags |= HF_SMM_MASK; kvm_vcpu_write_guest(vcpu, vcpu->arch.smbase + 0xfe00, buf, sizeof(buf)); - if (kvm_x86_ops->get_nmi_mask(vcpu)) + if (kvm_x86_ops_get_nmi_mask(vcpu)) vcpu->arch.hflags |= HF_SMM_INSIDE_NMI_MASK; else - kvm_x86_ops->set_nmi_mask(vcpu, true); + kvm_x86_ops_set_nmi_mask(vcpu, true); kvm_set_rflags(vcpu, X86_EFLAGS_FIXED); kvm_rip_write(vcpu, 0x8000); cr0 = vcpu->arch.cr0 & ~(X86_CR0_PE | X86_CR0_EM | X86_CR0_TS | X86_CR0_PG); - kvm_x86_ops->set_cr0(vcpu, cr0); + kvm_x86_ops_set_cr0(vcpu, cr0); vcpu->arch.cr0 = cr0; - kvm_x86_ops->set_cr4(vcpu, 0); + kvm_x86_ops_set_cr4(vcpu, 0); /* Undocumented: IDT limit is set to zero on entry to SMM. */ dt.address = dt.size = 0; - kvm_x86_ops->set_idt(vcpu, &dt); + kvm_x86_ops_set_idt(vcpu, &dt); __kvm_set_dr(vcpu, 7, DR7_FIXED_1); @@ -7761,7 +7761,7 @@ static void enter_smm(struct kvm_vcpu *vcpu) #ifdef CONFIG_X86_64 if (guest_cpuid_has(vcpu, X86_FEATURE_LM)) - kvm_x86_ops->set_efer(vcpu, 0); + kvm_x86_ops_set_efer(vcpu, 0); #endif kvm_update_cpuid(vcpu); @@ -7790,7 +7790,7 @@ static void vcpu_scan_ioapic(struct kvm_vcpu *vcpu) kvm_scan_ioapic_routes(vcpu, vcpu->arch.ioapic_handled_vectors); else { if (vcpu->arch.apicv_active) - kvm_x86_ops->sync_pir_to_irr(vcpu); + kvm_x86_ops_sync_pir_to_irr(vcpu); if (ioapic_in_kernel(vcpu->kvm)) kvm_ioapic_scan_entry(vcpu, vcpu->arch.ioapic_handled_vectors); } @@ -7810,7 +7810,7 @@ static void vcpu_load_eoi_exitmap(struct kvm_vcpu *vcpu) bitmap_or((ulong *)eoi_exit_bitmap, vcpu->arch.ioapic_handled_vectors, vcpu_to_synic(vcpu)->vec_bitmap, 256); - kvm_x86_ops->load_eoi_exitmap(vcpu, eoi_exit_bitmap); + kvm_x86_ops_load_eoi_exitmap(vcpu, eoi_exit_bitmap); } int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, @@ -7843,7 +7843,7 @@ void kvm_vcpu_reload_apic_access_page(struct kvm_vcpu *vcpu) page = gfn_to_page(vcpu->kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); if (is_error_page(page)) return; - kvm_x86_ops->set_apic_access_page_addr(vcpu, page_to_phys(page)); + kvm_x86_ops_set_apic_access_page_addr(vcpu, page_to_phys(page)); /* * Do not pin apic access page in memory, the MMU notifier @@ -7875,7 +7875,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) if (kvm_request_pending(vcpu)) { if (kvm_check_request(KVM_REQ_GET_VMCS12_PAGES, vcpu)) - kvm_x86_ops->get_vmcs12_pages(vcpu); + kvm_x86_ops_get_vmcs12_pages(vcpu); if (kvm_check_request(KVM_REQ_MMU_RELOAD, vcpu)) kvm_mmu_unload(vcpu); if (kvm_check_request(KVM_REQ_MIGRATE_TIMER, vcpu)) @@ -7993,12 +7993,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) * SMI. */ if (vcpu->arch.smi_pending && !is_smm(vcpu)) - if (!kvm_x86_ops->enable_smi_window(vcpu)) + if (!kvm_x86_ops_enable_smi_window(vcpu)) req_immediate_exit = true; if (vcpu->arch.nmi_pending) - kvm_x86_ops->enable_nmi_window(vcpu); + kvm_x86_ops_enable_nmi_window(vcpu); if (kvm_cpu_has_injectable_intr(vcpu) || req_int_win) - kvm_x86_ops->enable_irq_window(vcpu); + kvm_x86_ops_enable_irq_window(vcpu); WARN_ON(vcpu->arch.exception.pending); } @@ -8015,7 +8015,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) preempt_disable(); - kvm_x86_ops->prepare_guest_switch(vcpu); + kvm_x86_ops_prepare_guest_switch(vcpu); /* * Disable IRQs before setting IN_GUEST_MODE. Posted interrupt @@ -8046,7 +8046,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) * notified with kvm_vcpu_kick. */ if (kvm_lapic_enabled(vcpu) && vcpu->arch.apicv_active) - kvm_x86_ops->sync_pir_to_irr(vcpu); + kvm_x86_ops_sync_pir_to_irr(vcpu); if (vcpu->mode == EXITING_GUEST_MODE || kvm_request_pending(vcpu) || need_resched() || signal_pending(current)) { @@ -8061,7 +8061,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) if (req_immediate_exit) { kvm_make_request(KVM_REQ_EVENT, vcpu); - kvm_x86_ops->request_immediate_exit(vcpu); + kvm_x86_ops_request_immediate_exit(vcpu); } trace_kvm_entry(vcpu->vcpu_id); @@ -8080,7 +8080,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) vcpu->arch.switch_db_regs &= ~KVM_DEBUGREG_RELOAD; } - kvm_x86_ops->run(vcpu); + kvm_x86_ops_run(vcpu); /* * Do this here before restoring debug registers on the host. And @@ -8090,7 +8090,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) */ if (unlikely(vcpu->arch.switch_db_regs & KVM_DEBUGREG_WONT_EXIT)) { WARN_ON(vcpu->guest_debug & KVM_GUESTDBG_USE_HW_BP); - kvm_x86_ops->sync_dirty_debug_regs(vcpu); + kvm_x86_ops_sync_dirty_debug_regs(vcpu); kvm_update_dr0123(vcpu); kvm_update_dr6(vcpu); kvm_update_dr7(vcpu); @@ -8112,7 +8112,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) vcpu->mode = OUTSIDE_GUEST_MODE; smp_wmb(); - kvm_x86_ops->handle_exit_irqoff(vcpu); + kvm_x86_ops_handle_exit_irqoff(vcpu); /* * Consume any pending interrupts, including the possible source of @@ -8156,11 +8156,11 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) kvm_lapic_sync_from_vapic(vcpu); vcpu->arch.gpa_available = false; - r = kvm_x86_ops->handle_exit(vcpu); + r = kvm_x86_ops_handle_exit(vcpu); return r; cancel_injection: - kvm_x86_ops->cancel_injection(vcpu); + kvm_x86_ops_cancel_injection(vcpu); if (unlikely(vcpu->arch.apic_attention)) kvm_lapic_sync_from_vapic(vcpu); out: @@ -8170,13 +8170,13 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) static inline int vcpu_block(struct kvm *kvm, struct kvm_vcpu *vcpu) { if (!kvm_arch_vcpu_runnable(vcpu) && - (!kvm_x86_ops->pre_block || kvm_x86_ops->pre_block(vcpu) == 0)) { + (!kvm_x86_ops->pre_block || kvm_x86_ops_pre_block(vcpu) == 0)) { srcu_read_unlock(&kvm->srcu, vcpu->srcu_idx); kvm_vcpu_block(vcpu); vcpu->srcu_idx = srcu_read_lock(&kvm->srcu); if (kvm_x86_ops->post_block) - kvm_x86_ops->post_block(vcpu); + kvm_x86_ops_post_block(vcpu); if (!kvm_check_request(KVM_REQ_UNHALT, vcpu)) return 1; @@ -8204,7 +8204,7 @@ static inline int vcpu_block(struct kvm *kvm, struct kvm_vcpu *vcpu) static inline bool kvm_vcpu_running(struct kvm_vcpu *vcpu) { if (is_guest_mode(vcpu) && kvm_x86_ops->check_nested_events) - kvm_x86_ops->check_nested_events(vcpu, false); + kvm_x86_ops_check_nested_events(vcpu, false); return (vcpu->arch.mp_state == KVM_MP_STATE_RUNNABLE && !vcpu->arch.apf.halted); @@ -8549,10 +8549,10 @@ static void __get_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) kvm_get_segment(vcpu, &sregs->tr, VCPU_SREG_TR); kvm_get_segment(vcpu, &sregs->ldt, VCPU_SREG_LDTR); - kvm_x86_ops->get_idt(vcpu, &dt); + kvm_x86_ops_get_idt(vcpu, &dt); sregs->idt.limit = dt.size; sregs->idt.base = dt.address; - kvm_x86_ops->get_gdt(vcpu, &dt); + kvm_x86_ops_get_gdt(vcpu, &dt); sregs->gdt.limit = dt.size; sregs->gdt.base = dt.address; @@ -8693,10 +8693,10 @@ static int __set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) dt.size = sregs->idt.limit; dt.address = sregs->idt.base; - kvm_x86_ops->set_idt(vcpu, &dt); + kvm_x86_ops_set_idt(vcpu, &dt); dt.size = sregs->gdt.limit; dt.address = sregs->gdt.base; - kvm_x86_ops->set_gdt(vcpu, &dt); + kvm_x86_ops_set_gdt(vcpu, &dt); vcpu->arch.cr2 = sregs->cr2; mmu_reset_needed |= kvm_read_cr3(vcpu) != sregs->cr3; @@ -8706,16 +8706,16 @@ static int __set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) kvm_set_cr8(vcpu, sregs->cr8); mmu_reset_needed |= vcpu->arch.efer != sregs->efer; - kvm_x86_ops->set_efer(vcpu, sregs->efer); + kvm_x86_ops_set_efer(vcpu, sregs->efer); mmu_reset_needed |= kvm_read_cr0(vcpu) != sregs->cr0; - kvm_x86_ops->set_cr0(vcpu, sregs->cr0); + kvm_x86_ops_set_cr0(vcpu, sregs->cr0); vcpu->arch.cr0 = sregs->cr0; mmu_reset_needed |= kvm_read_cr4(vcpu) != sregs->cr4; cpuid_update_needed |= ((kvm_read_cr4(vcpu) ^ sregs->cr4) & (X86_CR4_OSXSAVE | X86_CR4_PKE)); - kvm_x86_ops->set_cr4(vcpu, sregs->cr4); + kvm_x86_ops_set_cr4(vcpu, sregs->cr4); if (cpuid_update_needed) kvm_update_cpuid(vcpu); @@ -8821,7 +8821,7 @@ int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu, */ kvm_set_rflags(vcpu, rflags); - kvm_x86_ops->update_bp_intercept(vcpu); + kvm_x86_ops_update_bp_intercept(vcpu); r = 0; @@ -8955,7 +8955,7 @@ void kvm_arch_vcpu_free(struct kvm_vcpu *vcpu) kvmclock_reset(vcpu); - kvm_x86_ops->vcpu_free(vcpu); + kvm_x86_ops_vcpu_free(vcpu); free_cpumask_var(wbinvd_dirty_mask); } @@ -8969,7 +8969,7 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, "kvm: SMP vm created on host with unstable TSC; " "guest TSC will not be reliable\n"); - vcpu = kvm_x86_ops->vcpu_create(kvm, id); + vcpu = kvm_x86_ops_vcpu_create(kvm, id); return vcpu; } @@ -9022,7 +9022,7 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu) kvm_mmu_unload(vcpu); vcpu_put(vcpu); - kvm_x86_ops->vcpu_free(vcpu); + kvm_x86_ops_vcpu_free(vcpu); } void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) @@ -9095,7 +9095,7 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) vcpu->arch.ia32_xss = 0; - kvm_x86_ops->vcpu_reset(vcpu, init_event); + kvm_x86_ops_vcpu_reset(vcpu, init_event); } void kvm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector) @@ -9120,7 +9120,7 @@ int kvm_arch_hardware_enable(void) bool stable, backwards_tsc = false; kvm_shared_msr_cpu_online(); - ret = kvm_x86_ops->hardware_enable(); + ret = kvm_x86_ops_hardware_enable(); if (ret != 0) return ret; @@ -9202,7 +9202,7 @@ int kvm_arch_hardware_enable(void) void kvm_arch_hardware_disable(void) { - kvm_x86_ops->hardware_disable(); + kvm_x86_ops_hardware_disable(); drop_user_return_notifiers(); } @@ -9210,7 +9210,7 @@ __init int kvm_arch_hardware_setup(void) { int r; - r = kvm_x86_ops->hardware_setup(); + r = kvm_x86_ops_hardware_setup(); if (r != 0) return r; @@ -9234,12 +9234,12 @@ __init int kvm_arch_hardware_setup(void) void kvm_arch_hardware_unsetup(void) { - kvm_x86_ops->hardware_unsetup(); + kvm_x86_ops_hardware_unsetup(); } __init int kvm_arch_check_processor_compat(void) { - return kvm_x86_ops->check_processor_compatibility(); + return kvm_x86_ops_check_processor_compatibility(); } bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu) @@ -9281,7 +9281,7 @@ int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu) goto fail_free_pio_data; if (irqchip_in_kernel(vcpu->kvm)) { - vcpu->arch.apicv_active = kvm_x86_ops->get_enable_apicv(vcpu); + vcpu->arch.apicv_active = kvm_x86_ops_get_enable_apicv(vcpu); r = kvm_create_lapic(vcpu, lapic_timer_advance_ns); if (r < 0) goto fail_mmu_destroy; @@ -9351,7 +9351,7 @@ void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu) { vcpu->arch.l1tf_flush_l1d = true; - kvm_x86_ops->sched_in(vcpu, cpu); + kvm_x86_ops_sched_in(vcpu, cpu); } int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) @@ -9386,7 +9386,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) kvm_page_track_init(kvm); kvm_mmu_init_vm(kvm); - return kvm_x86_ops->vm_init(kvm); + return kvm_x86_ops_vm_init(kvm); } static void kvm_unload_vcpu_mmu(struct kvm_vcpu *vcpu) @@ -9503,7 +9503,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm) x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, 0, 0); } if (kvm_x86_ops->vm_destroy) - kvm_x86_ops->vm_destroy(kvm); + kvm_x86_ops_vm_destroy(kvm); kvm_pic_destroy(kvm); kvm_ioapic_destroy(kvm); kvm_free_vcpus(kvm); @@ -9660,12 +9660,12 @@ static void kvm_mmu_slot_apply_flags(struct kvm *kvm, */ if (new->flags & KVM_MEM_LOG_DIRTY_PAGES) { if (kvm_x86_ops->slot_enable_log_dirty) - kvm_x86_ops->slot_enable_log_dirty(kvm, new); + kvm_x86_ops_slot_enable_log_dirty(kvm, new); else kvm_mmu_slot_remove_write_access(kvm, new); } else { if (kvm_x86_ops->slot_disable_log_dirty) - kvm_x86_ops->slot_disable_log_dirty(kvm, new); + kvm_x86_ops_slot_disable_log_dirty(kvm, new); } } @@ -9725,7 +9725,7 @@ static inline bool kvm_guest_apic_has_interrupt(struct kvm_vcpu *vcpu) { return (is_guest_mode(vcpu) && kvm_x86_ops->guest_apic_has_interrupt && - kvm_x86_ops->guest_apic_has_interrupt(vcpu)); + kvm_x86_ops_guest_apic_has_interrupt(vcpu)); } static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu) @@ -9744,7 +9744,7 @@ static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu) if (kvm_test_request(KVM_REQ_NMI, vcpu) || (vcpu->arch.nmi_pending && - kvm_x86_ops->nmi_allowed(vcpu))) + kvm_x86_ops_nmi_allowed(vcpu))) return true; if (kvm_test_request(KVM_REQ_SMI, vcpu) || @@ -9777,7 +9777,7 @@ bool kvm_arch_dy_runnable(struct kvm_vcpu *vcpu) kvm_test_request(KVM_REQ_EVENT, vcpu)) return true; - if (vcpu->arch.apicv_active && kvm_x86_ops->dy_apicv_has_pending_interrupt(vcpu)) + if (vcpu->arch.apicv_active && kvm_x86_ops_dy_apicv_has_pending_interrupt(vcpu)) return true; return false; @@ -9795,7 +9795,7 @@ int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) int kvm_arch_interrupt_allowed(struct kvm_vcpu *vcpu) { - return kvm_x86_ops->interrupt_allowed(vcpu); + return kvm_x86_ops_interrupt_allowed(vcpu); } unsigned long kvm_get_linear_rip(struct kvm_vcpu *vcpu) @@ -9817,7 +9817,7 @@ unsigned long kvm_get_rflags(struct kvm_vcpu *vcpu) { unsigned long rflags; - rflags = kvm_x86_ops->get_rflags(vcpu); + rflags = kvm_x86_ops_get_rflags(vcpu); if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP) rflags &= ~X86_EFLAGS_TF; return rflags; @@ -9829,7 +9829,7 @@ static void __kvm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags) if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP && kvm_is_linear_rip(vcpu, vcpu->arch.singlestep_rip)) rflags |= X86_EFLAGS_TF; - kvm_x86_ops->set_rflags(vcpu, rflags); + kvm_x86_ops_set_rflags(vcpu, rflags); } void kvm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags) @@ -9940,7 +9940,7 @@ static bool kvm_can_deliver_async_pf(struct kvm_vcpu *vcpu) if (!(vcpu->arch.apf.msr_val & KVM_ASYNC_PF_ENABLED) || (vcpu->arch.apf.send_user_only && - kvm_x86_ops->get_cpl(vcpu) == 0)) + kvm_x86_ops_get_cpl(vcpu) == 0)) return false; return true; @@ -9960,7 +9960,7 @@ bool kvm_can_do_async_pf(struct kvm_vcpu *vcpu) * If interrupts are off we cannot even use an artificial * halt state. */ - return kvm_x86_ops->interrupt_allowed(vcpu); + return kvm_x86_ops_interrupt_allowed(vcpu); } void kvm_arch_async_page_not_present(struct kvm_vcpu *vcpu, @@ -10089,7 +10089,7 @@ int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons, irqfd->producer = prod; - return kvm_x86_ops->update_pi_irte(irqfd->kvm, + return kvm_x86_ops_update_pi_irte(irqfd->kvm, prod->irq, irqfd->gsi, 1); } @@ -10109,7 +10109,7 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons, * when the irq is masked/disabled or the consumer side (KVM * int this case doesn't want to receive the interrupts. */ - ret = kvm_x86_ops->update_pi_irte(irqfd->kvm, prod->irq, irqfd->gsi, 0); + ret = kvm_x86_ops_update_pi_irte(irqfd->kvm, prod->irq, irqfd->gsi, 0); if (ret) printk(KERN_INFO "irq bypass consumer (token %p) unregistration" " fails: %d\n", irqfd->consumer.token, ret); @@ -10118,7 +10118,7 @@ void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons, int kvm_arch_update_irqfd_routing(struct kvm *kvm, unsigned int host_irq, uint32_t guest_irq, bool set) { - return kvm_x86_ops->update_pi_irte(kvm, host_irq, guest_irq, set); + return kvm_x86_ops_update_pi_irte(kvm, host_irq, guest_irq, set); } bool kvm_vector_hashing_enabled(void) diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index b5274e2a53cf..6e4e2d75f402 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -96,7 +96,7 @@ static inline bool is_64_bit_mode(struct kvm_vcpu *vcpu) if (!is_long_mode(vcpu)) return false; - kvm_x86_ops->get_cs_db_l_bits(vcpu, &cs_db, &cs_l); + kvm_x86_ops_get_cs_db_l_bits(vcpu, &cs_db, &cs_l); return cs_l; } From patchwork Fri Sep 20 21:25:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154871 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8875C15E6 for ; Fri, 20 Sep 2019 21:26:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 523CB208C3 for ; Fri, 20 Sep 2019 21:26:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393948AbfITVZ4 (ORCPT ); Fri, 20 Sep 2019 17:25:56 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33936 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393440AbfITVZO (ORCPT ); Fri, 20 Sep 2019 17:25:14 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AD8288A1C87; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 51CDA1001947; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 11/17] KVM: monolithic: x86: remove exports Date: Fri, 20 Sep 2019 17:25:03 -0400 Message-Id: <20190920212509.2578-12-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.69]); Fri, 20 Sep 2019 21:25:13 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The exports would be duplicated across kvm-amd and kvm-intel if they're kept, this cleanup various harmless warnings about it. Signed-off-by: Andrea Arcangeli --- arch/x86/kvm/cpuid.c | 5 -- arch/x86/kvm/hyperv.c | 2 - arch/x86/kvm/irq.c | 4 -- arch/x86/kvm/irq_comm.c | 2 - arch/x86/kvm/lapic.c | 16 ------- arch/x86/kvm/mmu.c | 24 ---------- arch/x86/kvm/mtrr.c | 2 - arch/x86/kvm/pmu.c | 3 -- arch/x86/kvm/x86.c | 101 ---------------------------------------- 9 files changed, 159 deletions(-) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index d8d840fc703e..e4fa9c629b0b 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -50,7 +50,6 @@ bool kvm_mpx_supported(void) return ((host_xcr0 & (XFEATURE_MASK_BNDREGS | XFEATURE_MASK_BNDCSR)) && kvm_x86_ops_mpx_supported()); } -EXPORT_SYMBOL_GPL(kvm_mpx_supported); u64 kvm_supported_xcr0(void) { @@ -192,7 +191,6 @@ int cpuid_query_maxphyaddr(struct kvm_vcpu *vcpu) not_found: return 36; } -EXPORT_SYMBOL_GPL(cpuid_query_maxphyaddr); /* when an old userspace process fills a new kernel module */ int kvm_vcpu_ioctl_set_cpuid(struct kvm_vcpu *vcpu, @@ -960,7 +958,6 @@ struct kvm_cpuid_entry2 *kvm_find_cpuid_entry(struct kvm_vcpu *vcpu, } return best; } -EXPORT_SYMBOL_GPL(kvm_find_cpuid_entry); /* * If no match is found, check whether we exceed the vCPU's limit @@ -1011,7 +1008,6 @@ bool kvm_cpuid(struct kvm_vcpu *vcpu, u32 *eax, u32 *ebx, trace_kvm_cpuid(function, *eax, *ebx, *ecx, *edx, entry_found); return entry_found; } -EXPORT_SYMBOL_GPL(kvm_cpuid); int kvm_emulate_cpuid(struct kvm_vcpu *vcpu) { @@ -1029,4 +1025,3 @@ int kvm_emulate_cpuid(struct kvm_vcpu *vcpu) kvm_rdx_write(vcpu, edx); return kvm_skip_emulated_instruction(vcpu); } -EXPORT_SYMBOL_GPL(kvm_emulate_cpuid); diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index e5177c5c126c..d875d60339c7 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -711,7 +711,6 @@ bool kvm_hv_assist_page_enabled(struct kvm_vcpu *vcpu) return false; return vcpu->arch.pv_eoi.msr_val & KVM_MSR_ENABLED; } -EXPORT_SYMBOL_GPL(kvm_hv_assist_page_enabled); bool kvm_hv_get_assist_page(struct kvm_vcpu *vcpu, struct hv_vp_assist_page *assist_page) @@ -721,7 +720,6 @@ bool kvm_hv_get_assist_page(struct kvm_vcpu *vcpu, return !kvm_read_guest_cached(vcpu->kvm, &vcpu->arch.pv_eoi.data, assist_page, sizeof(*assist_page)); } -EXPORT_SYMBOL_GPL(kvm_hv_get_assist_page); static void stimer_prepare_msg(struct kvm_vcpu_hv_stimer *stimer) { diff --git a/arch/x86/kvm/irq.c b/arch/x86/kvm/irq.c index e330e7d125f7..ba4300f36a32 100644 --- a/arch/x86/kvm/irq.c +++ b/arch/x86/kvm/irq.c @@ -26,7 +26,6 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu) return 0; } -EXPORT_SYMBOL(kvm_cpu_has_pending_timer); /* * check if there is a pending userspace external interrupt @@ -109,7 +108,6 @@ int kvm_cpu_has_interrupt(struct kvm_vcpu *v) return kvm_apic_has_interrupt(v) != -1; /* LAPIC */ } -EXPORT_SYMBOL_GPL(kvm_cpu_has_interrupt); /* * Read pending interrupt(from non-APIC source) @@ -146,14 +144,12 @@ int kvm_cpu_get_interrupt(struct kvm_vcpu *v) return kvm_get_apic_interrupt(v); /* APIC */ } -EXPORT_SYMBOL_GPL(kvm_cpu_get_interrupt); void kvm_inject_pending_timer_irqs(struct kvm_vcpu *vcpu) { if (lapic_in_kernel(vcpu)) kvm_inject_apic_timer_irqs(vcpu); } -EXPORT_SYMBOL_GPL(kvm_inject_pending_timer_irqs); void __kvm_migrate_timers(struct kvm_vcpu *vcpu) { diff --git a/arch/x86/kvm/irq_comm.c b/arch/x86/kvm/irq_comm.c index 8ecd48d31800..64a13d5fcc9f 100644 --- a/arch/x86/kvm/irq_comm.c +++ b/arch/x86/kvm/irq_comm.c @@ -122,7 +122,6 @@ void kvm_set_msi_irq(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e, irq->level = 1; irq->shorthand = 0; } -EXPORT_SYMBOL_GPL(kvm_set_msi_irq); static inline bool kvm_msi_route_invalid(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e) @@ -346,7 +345,6 @@ bool kvm_intr_is_single_vcpu(struct kvm *kvm, struct kvm_lapic_irq *irq, return r == 1; } -EXPORT_SYMBOL_GPL(kvm_intr_is_single_vcpu); #define IOAPIC_ROUTING_ENTRY(irq) \ { .gsi = irq, .type = KVM_IRQ_ROUTING_IRQCHIP, \ diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index d8dce6d40174..d32840fe45f7 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -122,7 +122,6 @@ bool kvm_can_post_timer_interrupt(struct kvm_vcpu *vcpu) { return pi_inject_timer && kvm_vcpu_apicv_active(vcpu); } -EXPORT_SYMBOL_GPL(kvm_can_post_timer_interrupt); static bool kvm_use_posted_timer_interrupt(struct kvm_vcpu *vcpu) { @@ -412,7 +411,6 @@ bool __kvm_apic_update_irr(u32 *pir, void *regs, int *max_irr) return ((max_updated_irr != -1) && (max_updated_irr == *max_irr)); } -EXPORT_SYMBOL_GPL(__kvm_apic_update_irr); bool kvm_apic_update_irr(struct kvm_vcpu *vcpu, u32 *pir, int *max_irr) { @@ -420,7 +418,6 @@ bool kvm_apic_update_irr(struct kvm_vcpu *vcpu, u32 *pir, int *max_irr) return __kvm_apic_update_irr(pir, apic->regs, max_irr); } -EXPORT_SYMBOL_GPL(kvm_apic_update_irr); static inline int apic_search_irr(struct kvm_lapic *apic) { @@ -544,7 +541,6 @@ int kvm_lapic_find_highest_irr(struct kvm_vcpu *vcpu) */ return apic_find_highest_irr(vcpu->arch.apic); } -EXPORT_SYMBOL_GPL(kvm_lapic_find_highest_irr); static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode, int vector, int level, int trig_mode, @@ -712,7 +708,6 @@ void kvm_apic_update_ppr(struct kvm_vcpu *vcpu) { apic_update_ppr(vcpu->arch.apic); } -EXPORT_SYMBOL_GPL(kvm_apic_update_ppr); static void apic_set_tpr(struct kvm_lapic *apic, u32 tpr) { @@ -823,7 +818,6 @@ bool kvm_apic_match_dest(struct kvm_vcpu *vcpu, struct kvm_lapic *source, return false; } } -EXPORT_SYMBOL_GPL(kvm_apic_match_dest); int kvm_vector_to_index(u32 vector, u32 dest_vcpus, const unsigned long *bitmap, u32 bitmap_size) @@ -1196,7 +1190,6 @@ void kvm_apic_set_eoi_accelerated(struct kvm_vcpu *vcpu, int vector) kvm_ioapic_send_eoi(apic, vector); kvm_make_request(KVM_REQ_EVENT, apic->vcpu); } -EXPORT_SYMBOL_GPL(kvm_apic_set_eoi_accelerated); static void apic_send_ipi(struct kvm_lapic *apic, u32 icr_low, u32 icr_high) { @@ -1355,7 +1348,6 @@ int kvm_lapic_reg_read(struct kvm_lapic *apic, u32 offset, int len, } return 0; } -EXPORT_SYMBOL_GPL(kvm_lapic_reg_read); static int apic_mmio_in_range(struct kvm_lapic *apic, gpa_t addr) { @@ -1533,7 +1525,6 @@ void kvm_wait_lapic_expire(struct kvm_vcpu *vcpu) if (lapic_timer_int_injected(vcpu)) __kvm_wait_lapic_expire(vcpu); } -EXPORT_SYMBOL_GPL(kvm_wait_lapic_expire); static void kvm_apic_inject_pending_timer_irqs(struct kvm_lapic *apic) { @@ -1698,7 +1689,6 @@ bool kvm_lapic_hv_timer_in_use(struct kvm_vcpu *vcpu) return vcpu->arch.apic->lapic_timer.hv_timer_in_use; } -EXPORT_SYMBOL_GPL(kvm_lapic_hv_timer_in_use); static void cancel_hv_timer(struct kvm_lapic *apic) { @@ -1799,13 +1789,11 @@ void kvm_lapic_expired_hv_timer(struct kvm_vcpu *vcpu) out: preempt_enable(); } -EXPORT_SYMBOL_GPL(kvm_lapic_expired_hv_timer); void kvm_lapic_switch_to_hv_timer(struct kvm_vcpu *vcpu) { restart_apic_timer(vcpu->arch.apic); } -EXPORT_SYMBOL_GPL(kvm_lapic_switch_to_hv_timer); void kvm_lapic_switch_to_sw_timer(struct kvm_vcpu *vcpu) { @@ -1817,7 +1805,6 @@ void kvm_lapic_switch_to_sw_timer(struct kvm_vcpu *vcpu) start_sw_timer(apic); preempt_enable(); } -EXPORT_SYMBOL_GPL(kvm_lapic_switch_to_sw_timer); void kvm_lapic_restart_hv_timer(struct kvm_vcpu *vcpu) { @@ -1987,7 +1974,6 @@ int kvm_lapic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val) return ret; } -EXPORT_SYMBOL_GPL(kvm_lapic_reg_write); static int apic_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this, gpa_t address, int len, const void *data) @@ -2026,7 +2012,6 @@ void kvm_lapic_set_eoi(struct kvm_vcpu *vcpu) { kvm_lapic_reg_write(vcpu->arch.apic, APIC_EOI, 0); } -EXPORT_SYMBOL_GPL(kvm_lapic_set_eoi); /* emulate APIC access in a trap manner */ void kvm_apic_write_nodecode(struct kvm_vcpu *vcpu, u32 offset) @@ -2041,7 +2026,6 @@ void kvm_apic_write_nodecode(struct kvm_vcpu *vcpu, u32 offset) /* TODO: optimize to just emulate side effect w/o one more write */ kvm_lapic_reg_write(vcpu->arch.apic, offset, val); } -EXPORT_SYMBOL_GPL(kvm_apic_write_nodecode); void kvm_free_lapic(struct kvm_vcpu *vcpu) { diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index affeba031937..bb2d05eb9271 100644 --- a/arch/x86/kvm/mmu.c +++ b/arch/x86/kvm/mmu.c @@ -308,7 +308,6 @@ void kvm_mmu_set_mmio_spte_mask(u64 mmio_mask, u64 mmio_value, u64 access_mask) shadow_mmio_mask = mmio_mask | SPTE_SPECIAL_MASK; shadow_mmio_access_mask = access_mask; } -EXPORT_SYMBOL_GPL(kvm_mmu_set_mmio_spte_mask); static bool is_mmio_spte(u64 spte) { @@ -474,7 +473,6 @@ void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask, shadow_acc_track_mask = acc_track_mask; shadow_me_mask = me_mask; } -EXPORT_SYMBOL_GPL(kvm_mmu_set_mask_ptes); static u8 kvm_get_shadow_phys_bits(void) { @@ -1702,7 +1700,6 @@ void kvm_mmu_clear_dirty_pt_masked(struct kvm *kvm, mask &= mask - 1; } } -EXPORT_SYMBOL_GPL(kvm_mmu_clear_dirty_pt_masked); /** * kvm_arch_mmu_enable_log_dirty_pt_masked - enable dirty logging for selected @@ -2853,7 +2850,6 @@ int kvm_mmu_unprotect_page(struct kvm *kvm, gfn_t gfn) return r; } -EXPORT_SYMBOL_GPL(kvm_mmu_unprotect_page); static void kvm_unsync_page(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp) { @@ -3621,7 +3617,6 @@ void kvm_mmu_free_roots(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, kvm_mmu_commit_zap_page(vcpu->kvm, &invalid_list); spin_unlock(&vcpu->kvm->mmu_lock); } -EXPORT_SYMBOL_GPL(kvm_mmu_free_roots); static int mmu_check_root(struct kvm_vcpu *vcpu, gfn_t root_gfn) { @@ -3846,7 +3841,6 @@ void kvm_mmu_sync_roots(struct kvm_vcpu *vcpu) kvm_mmu_audit(vcpu, AUDIT_POST_SYNC); spin_unlock(&vcpu->kvm->mmu_lock); } -EXPORT_SYMBOL_GPL(kvm_mmu_sync_roots); static gpa_t nonpaging_gva_to_gpa(struct kvm_vcpu *vcpu, gva_t vaddr, u32 access, struct x86_exception *exception) @@ -4114,7 +4108,6 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 error_code, } return r; } -EXPORT_SYMBOL_GPL(kvm_handle_page_fault); static bool check_hugepage_cache_consistency(struct kvm_vcpu *vcpu, gfn_t gfn, int level) @@ -4294,7 +4287,6 @@ void kvm_mmu_new_cr3(struct kvm_vcpu *vcpu, gpa_t new_cr3, bool skip_tlb_flush) __kvm_mmu_new_cr3(vcpu, new_cr3, kvm_mmu_calc_root_page_role(vcpu), skip_tlb_flush); } -EXPORT_SYMBOL_GPL(kvm_mmu_new_cr3); static unsigned long get_cr3(struct kvm_vcpu *vcpu) { @@ -4534,7 +4526,6 @@ reset_shadow_zero_bits_mask(struct kvm_vcpu *vcpu, struct kvm_mmu *context) } } -EXPORT_SYMBOL_GPL(reset_shadow_zero_bits_mask); static inline bool boot_cpu_is_amd(void) { @@ -4954,7 +4945,6 @@ void kvm_init_shadow_mmu(struct kvm_vcpu *vcpu) context->mmu_role.as_u64 = new_role.as_u64; reset_shadow_zero_bits_mask(vcpu, context); } -EXPORT_SYMBOL_GPL(kvm_init_shadow_mmu); static union kvm_mmu_role kvm_calc_shadow_ept_root_page_role(struct kvm_vcpu *vcpu, bool accessed_dirty, @@ -5018,7 +5008,6 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, bool execonly, reset_rsvds_bits_mask_ept(vcpu, context, execonly); reset_ept_shadow_zero_bits_mask(vcpu, context, execonly); } -EXPORT_SYMBOL_GPL(kvm_init_shadow_ept_mmu); static void init_kvm_softmmu(struct kvm_vcpu *vcpu) { @@ -5098,7 +5087,6 @@ void kvm_init_mmu(struct kvm_vcpu *vcpu, bool reset_roots) else init_kvm_softmmu(vcpu); } -EXPORT_SYMBOL_GPL(kvm_init_mmu); static union kvm_mmu_page_role kvm_mmu_calc_root_page_role(struct kvm_vcpu *vcpu) @@ -5118,7 +5106,6 @@ void kvm_mmu_reset_context(struct kvm_vcpu *vcpu) kvm_mmu_unload(vcpu); kvm_init_mmu(vcpu, true); } -EXPORT_SYMBOL_GPL(kvm_mmu_reset_context); int kvm_mmu_load(struct kvm_vcpu *vcpu) { @@ -5136,7 +5123,6 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu) out: return r; } -EXPORT_SYMBOL_GPL(kvm_mmu_load); void kvm_mmu_unload(struct kvm_vcpu *vcpu) { @@ -5145,7 +5131,6 @@ void kvm_mmu_unload(struct kvm_vcpu *vcpu) kvm_mmu_free_roots(vcpu, &vcpu->arch.guest_mmu, KVM_MMU_ROOTS_ALL); WARN_ON(VALID_PAGE(vcpu->arch.guest_mmu.root_hpa)); } -EXPORT_SYMBOL_GPL(kvm_mmu_unload); static void mmu_pte_write_new_pte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, u64 *spte, @@ -5357,7 +5342,6 @@ int kvm_mmu_unprotect_page_virt(struct kvm_vcpu *vcpu, gva_t gva) return r; } -EXPORT_SYMBOL_GPL(kvm_mmu_unprotect_page_virt); static int make_mmu_pages_available(struct kvm_vcpu *vcpu) { @@ -5464,7 +5448,6 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u64 error_code, BUG(); } } -EXPORT_SYMBOL_GPL(kvm_mmu_page_fault); void kvm_mmu_invlpg(struct kvm_vcpu *vcpu, gva_t gva) { @@ -5495,7 +5478,6 @@ void kvm_mmu_invlpg(struct kvm_vcpu *vcpu, gva_t gva) kvm_x86_ops_tlb_flush_gva(vcpu, gva); ++vcpu->stat.invlpg; } -EXPORT_SYMBOL_GPL(kvm_mmu_invlpg); void kvm_mmu_invpcid_gva(struct kvm_vcpu *vcpu, gva_t gva, unsigned long pcid) { @@ -5527,19 +5509,16 @@ void kvm_mmu_invpcid_gva(struct kvm_vcpu *vcpu, gva_t gva, unsigned long pcid) * for them. */ } -EXPORT_SYMBOL_GPL(kvm_mmu_invpcid_gva); void kvm_enable_tdp(void) { tdp_enabled = true; } -EXPORT_SYMBOL_GPL(kvm_enable_tdp); void kvm_disable_tdp(void) { tdp_enabled = false; } -EXPORT_SYMBOL_GPL(kvm_disable_tdp); /* The return value indicates if tlb flush on all vcpus is needed. */ @@ -5920,7 +5899,6 @@ void kvm_mmu_slot_leaf_clear_dirty(struct kvm *kvm, kvm_flush_remote_tlbs_with_address(kvm, memslot->base_gfn, memslot->npages); } -EXPORT_SYMBOL_GPL(kvm_mmu_slot_leaf_clear_dirty); void kvm_mmu_slot_largepage_remove_write_access(struct kvm *kvm, struct kvm_memory_slot *memslot) @@ -5939,7 +5917,6 @@ void kvm_mmu_slot_largepage_remove_write_access(struct kvm *kvm, kvm_flush_remote_tlbs_with_address(kvm, memslot->base_gfn, memslot->npages); } -EXPORT_SYMBOL_GPL(kvm_mmu_slot_largepage_remove_write_access); void kvm_mmu_slot_set_dirty(struct kvm *kvm, struct kvm_memory_slot *memslot) @@ -5957,7 +5934,6 @@ void kvm_mmu_slot_set_dirty(struct kvm *kvm, kvm_flush_remote_tlbs_with_address(kvm, memslot->base_gfn, memslot->npages); } -EXPORT_SYMBOL_GPL(kvm_mmu_slot_set_dirty); static void __kvm_mmu_zap_all(struct kvm *kvm, bool mmio_only) { diff --git a/arch/x86/kvm/mtrr.c b/arch/x86/kvm/mtrr.c index 25ce3edd1872..477f7141f793 100644 --- a/arch/x86/kvm/mtrr.c +++ b/arch/x86/kvm/mtrr.c @@ -91,7 +91,6 @@ bool kvm_mtrr_valid(struct kvm_vcpu *vcpu, u32 msr, u64 data) return true; } -EXPORT_SYMBOL_GPL(kvm_mtrr_valid); static bool mtrr_is_enabled(struct kvm_mtrr *mtrr_state) { @@ -686,7 +685,6 @@ u8 kvm_mtrr_get_guest_memory_type(struct kvm_vcpu *vcpu, gfn_t gfn) return type; } -EXPORT_SYMBOL_GPL(kvm_mtrr_get_guest_memory_type); bool kvm_mtrr_check_gfn_range_consistency(struct kvm_vcpu *vcpu, gfn_t gfn, int page_num) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 363d3f86fb99..f87546e1d97d 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -200,7 +200,6 @@ void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel) (eventsel & HSW_IN_TX), (eventsel & HSW_IN_TX_CHECKPOINTED)); } -EXPORT_SYMBOL_GPL(reprogram_gp_counter); void reprogram_fixed_counter(struct kvm_pmc *pmc, u8 ctrl, int idx) { @@ -230,7 +229,6 @@ void reprogram_fixed_counter(struct kvm_pmc *pmc, u8 ctrl, int idx) !(en_field & 0x1), /* exclude kernel */ pmi, false, false); } -EXPORT_SYMBOL_GPL(reprogram_fixed_counter); void reprogram_counter(struct kvm_pmu *pmu, int pmc_idx) { @@ -248,7 +246,6 @@ void reprogram_counter(struct kvm_pmu *pmu, int pmc_idx) reprogram_fixed_counter(pmc, ctrl, idx); } } -EXPORT_SYMBOL_GPL(reprogram_counter); void kvm_pmu_handle_event(struct kvm_vcpu *vcpu) { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 2db3c0cb2631..a6b441096d48 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -76,7 +76,6 @@ #define MAX_IO_MSRS 256 #define KVM_MAX_MCE_BANKS 32 u64 __read_mostly kvm_mce_cap_supported = MCG_CTL_P | MCG_SER_P; -EXPORT_SYMBOL_GPL(kvm_mce_cap_supported); #define emul_to_vcpu(ctxt) \ container_of(ctxt, struct kvm_vcpu, arch.emulate_ctxt) @@ -106,7 +105,6 @@ static void store_regs(struct kvm_vcpu *vcpu); static int sync_regs(struct kvm_vcpu *vcpu); struct kvm_x86_ops *kvm_x86_ops __read_mostly; -EXPORT_SYMBOL_GPL(kvm_x86_ops); static bool __read_mostly ignore_msrs = 0; module_param(ignore_msrs, bool, S_IRUGO | S_IWUSR); @@ -121,15 +119,10 @@ static bool __read_mostly kvmclock_periodic_sync = true; module_param(kvmclock_periodic_sync, bool, S_IRUGO); bool __read_mostly kvm_has_tsc_control; -EXPORT_SYMBOL_GPL(kvm_has_tsc_control); u32 __read_mostly kvm_max_guest_tsc_khz; -EXPORT_SYMBOL_GPL(kvm_max_guest_tsc_khz); u8 __read_mostly kvm_tsc_scaling_ratio_frac_bits; -EXPORT_SYMBOL_GPL(kvm_tsc_scaling_ratio_frac_bits); u64 __read_mostly kvm_max_tsc_scaling_ratio; -EXPORT_SYMBOL_GPL(kvm_max_tsc_scaling_ratio); u64 __read_mostly kvm_default_tsc_scaling_ratio; -EXPORT_SYMBOL_GPL(kvm_default_tsc_scaling_ratio); /* tsc tolerance in parts per million - default to 1/2 of the NTP threshold */ static u32 __read_mostly tsc_tolerance_ppm = 250; @@ -149,7 +142,6 @@ module_param(vector_hashing, bool, S_IRUGO); bool __read_mostly enable_vmware_backdoor = false; module_param(enable_vmware_backdoor, bool, S_IRUGO); -EXPORT_SYMBOL_GPL(enable_vmware_backdoor); static bool __read_mostly force_emulation_prefix = false; module_param(force_emulation_prefix, bool, S_IRUGO); @@ -221,7 +213,6 @@ struct kvm_stats_debugfs_item debugfs_entries[] = { u64 __read_mostly host_xcr0; struct kmem_cache *x86_fpu_cache; -EXPORT_SYMBOL_GPL(x86_fpu_cache); static int emulator_fix_hypercall(struct x86_emulate_ctxt *ctxt); @@ -283,7 +274,6 @@ void kvm_define_shared_msr(unsigned slot, u32 msr) if (slot >= shared_msrs_global.nr) shared_msrs_global.nr = slot + 1; } -EXPORT_SYMBOL_GPL(kvm_define_shared_msr); static void kvm_shared_msr_cpu_online(void) { @@ -313,7 +303,6 @@ int kvm_set_shared_msr(unsigned slot, u64 value, u64 mask) } return 0; } -EXPORT_SYMBOL_GPL(kvm_set_shared_msr); static void drop_user_return_notifiers(void) { @@ -328,13 +317,11 @@ u64 kvm_get_apic_base(struct kvm_vcpu *vcpu) { return vcpu->arch.apic_base; } -EXPORT_SYMBOL_GPL(kvm_get_apic_base); enum lapic_mode kvm_get_apic_mode(struct kvm_vcpu *vcpu) { return kvm_apic_mode(kvm_get_apic_base(vcpu)); } -EXPORT_SYMBOL_GPL(kvm_get_apic_mode); int kvm_set_apic_base(struct kvm_vcpu *vcpu, struct msr_data *msr_info) { @@ -355,14 +342,12 @@ int kvm_set_apic_base(struct kvm_vcpu *vcpu, struct msr_data *msr_info) kvm_lapic_set_base(vcpu, msr_info->data); return 0; } -EXPORT_SYMBOL_GPL(kvm_set_apic_base); asmlinkage __visible void kvm_spurious_fault(void) { /* Fault while not rebooting. We want the trace. */ BUG(); } -EXPORT_SYMBOL_GPL(kvm_spurious_fault); #define EXCPT_BENIGN 0 #define EXCPT_CONTRIBUTORY 1 @@ -450,7 +435,6 @@ void kvm_deliver_exception_payload(struct kvm_vcpu *vcpu) vcpu->arch.exception.has_payload = false; vcpu->arch.exception.payload = 0; } -EXPORT_SYMBOL_GPL(kvm_deliver_exception_payload); static void kvm_multiple_exception(struct kvm_vcpu *vcpu, unsigned nr, bool has_error, u32 error_code, @@ -544,13 +528,11 @@ void kvm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr) { kvm_multiple_exception(vcpu, nr, false, 0, false, 0, false); } -EXPORT_SYMBOL_GPL(kvm_queue_exception); void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned nr) { kvm_multiple_exception(vcpu, nr, false, 0, false, 0, true); } -EXPORT_SYMBOL_GPL(kvm_requeue_exception); static void kvm_queue_exception_p(struct kvm_vcpu *vcpu, unsigned nr, unsigned long payload) @@ -574,7 +556,6 @@ int kvm_complete_insn_gp(struct kvm_vcpu *vcpu, int err) return 1; } -EXPORT_SYMBOL_GPL(kvm_complete_insn_gp); void kvm_inject_page_fault(struct kvm_vcpu *vcpu, struct x86_exception *fault) { @@ -589,7 +570,6 @@ void kvm_inject_page_fault(struct kvm_vcpu *vcpu, struct x86_exception *fault) fault->address); } } -EXPORT_SYMBOL_GPL(kvm_inject_page_fault); static bool kvm_propagate_fault(struct kvm_vcpu *vcpu, struct x86_exception *fault) { @@ -606,19 +586,16 @@ void kvm_inject_nmi(struct kvm_vcpu *vcpu) atomic_inc(&vcpu->arch.nmi_queued); kvm_make_request(KVM_REQ_NMI, vcpu); } -EXPORT_SYMBOL_GPL(kvm_inject_nmi); void kvm_queue_exception_e(struct kvm_vcpu *vcpu, unsigned nr, u32 error_code) { kvm_multiple_exception(vcpu, nr, true, error_code, false, 0, false); } -EXPORT_SYMBOL_GPL(kvm_queue_exception_e); void kvm_requeue_exception_e(struct kvm_vcpu *vcpu, unsigned nr, u32 error_code) { kvm_multiple_exception(vcpu, nr, true, error_code, false, 0, true); } -EXPORT_SYMBOL_GPL(kvm_requeue_exception_e); /* * Checks if cpl <= required_cpl; if true, return true. Otherwise queue @@ -631,7 +608,6 @@ bool kvm_require_cpl(struct kvm_vcpu *vcpu, int required_cpl) kvm_queue_exception_e(vcpu, GP_VECTOR, 0); return false; } -EXPORT_SYMBOL_GPL(kvm_require_cpl); bool kvm_require_dr(struct kvm_vcpu *vcpu, int dr) { @@ -641,7 +617,6 @@ bool kvm_require_dr(struct kvm_vcpu *vcpu, int dr) kvm_queue_exception(vcpu, UD_VECTOR); return false; } -EXPORT_SYMBOL_GPL(kvm_require_dr); /* * This function will be used to read from the physical memory of the currently @@ -665,7 +640,6 @@ int kvm_read_guest_page_mmu(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, return kvm_vcpu_read_guest_page(vcpu, real_gfn, data, offset, len); } -EXPORT_SYMBOL_GPL(kvm_read_guest_page_mmu); static int kvm_read_nested_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data, int offset, int len, u32 access) @@ -716,7 +690,6 @@ int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3) return ret; } -EXPORT_SYMBOL_GPL(load_pdptrs); bool pdptrs_changed(struct kvm_vcpu *vcpu) { @@ -744,7 +717,6 @@ bool pdptrs_changed(struct kvm_vcpu *vcpu) return changed; } -EXPORT_SYMBOL_GPL(pdptrs_changed); int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) { @@ -803,13 +775,11 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) return 0; } -EXPORT_SYMBOL_GPL(kvm_set_cr0); void kvm_lmsw(struct kvm_vcpu *vcpu, unsigned long msw) { (void)kvm_set_cr0(vcpu, kvm_read_cr0_bits(vcpu, ~0x0eul) | (msw & 0x0f)); } -EXPORT_SYMBOL_GPL(kvm_lmsw); void kvm_load_guest_xcr0(struct kvm_vcpu *vcpu) { @@ -821,7 +791,6 @@ void kvm_load_guest_xcr0(struct kvm_vcpu *vcpu) vcpu->guest_xcr0_loaded = 1; } } -EXPORT_SYMBOL_GPL(kvm_load_guest_xcr0); void kvm_put_guest_xcr0(struct kvm_vcpu *vcpu) { @@ -831,7 +800,6 @@ void kvm_put_guest_xcr0(struct kvm_vcpu *vcpu) vcpu->guest_xcr0_loaded = 0; } } -EXPORT_SYMBOL_GPL(kvm_put_guest_xcr0); static int __kvm_set_xcr(struct kvm_vcpu *vcpu, u32 index, u64 xcr) { @@ -882,7 +850,6 @@ int kvm_set_xcr(struct kvm_vcpu *vcpu, u32 index, u64 xcr) } return 0; } -EXPORT_SYMBOL_GPL(kvm_set_xcr); int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) { @@ -944,7 +911,6 @@ int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) return 0; } -EXPORT_SYMBOL_GPL(kvm_set_cr4); int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) { @@ -979,7 +945,6 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) return 0; } -EXPORT_SYMBOL_GPL(kvm_set_cr3); int kvm_set_cr8(struct kvm_vcpu *vcpu, unsigned long cr8) { @@ -991,7 +956,6 @@ int kvm_set_cr8(struct kvm_vcpu *vcpu, unsigned long cr8) vcpu->arch.cr8 = cr8; return 0; } -EXPORT_SYMBOL_GPL(kvm_set_cr8); unsigned long kvm_get_cr8(struct kvm_vcpu *vcpu) { @@ -1000,7 +964,6 @@ unsigned long kvm_get_cr8(struct kvm_vcpu *vcpu) else return vcpu->arch.cr8; } -EXPORT_SYMBOL_GPL(kvm_get_cr8); static void kvm_update_dr0123(struct kvm_vcpu *vcpu) { @@ -1079,7 +1042,6 @@ int kvm_set_dr(struct kvm_vcpu *vcpu, int dr, unsigned long val) } return 0; } -EXPORT_SYMBOL_GPL(kvm_set_dr); int kvm_get_dr(struct kvm_vcpu *vcpu, int dr, unsigned long *val) { @@ -1103,7 +1065,6 @@ int kvm_get_dr(struct kvm_vcpu *vcpu, int dr, unsigned long *val) } return 0; } -EXPORT_SYMBOL_GPL(kvm_get_dr); bool kvm_rdpmc(struct kvm_vcpu *vcpu) { @@ -1118,7 +1079,6 @@ bool kvm_rdpmc(struct kvm_vcpu *vcpu) kvm_rdx_write(vcpu, data >> 32); return err; } -EXPORT_SYMBOL_GPL(kvm_rdpmc); /* * List of msr numbers which we expose to userspace through KVM_GET_MSRS @@ -1325,7 +1285,6 @@ bool kvm_valid_efer(struct kvm_vcpu *vcpu, u64 efer) return __kvm_valid_efer(vcpu, efer); } -EXPORT_SYMBOL_GPL(kvm_valid_efer); static int set_efer(struct kvm_vcpu *vcpu, struct msr_data *msr_info) { @@ -1360,7 +1319,6 @@ void kvm_enable_efer_bits(u64 mask) { efer_reserved_bits &= ~mask; } -EXPORT_SYMBOL_GPL(kvm_enable_efer_bits); /* * Write @data into the MSR specified by @index. Select MSR specific fault @@ -1771,7 +1729,6 @@ u64 kvm_scale_tsc(struct kvm_vcpu *vcpu, u64 tsc) return _tsc; } -EXPORT_SYMBOL_GPL(kvm_scale_tsc); static u64 kvm_compute_tsc_offset(struct kvm_vcpu *vcpu, u64 target_tsc) { @@ -1788,7 +1745,6 @@ u64 kvm_read_l1_tsc(struct kvm_vcpu *vcpu, u64 host_tsc) return tsc_offset + kvm_scale_tsc(vcpu, host_tsc); } -EXPORT_SYMBOL_GPL(kvm_read_l1_tsc); static void kvm_vcpu_write_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) { @@ -1911,7 +1867,6 @@ void kvm_write_tsc(struct kvm_vcpu *vcpu, struct msr_data *msr) spin_unlock(&kvm->arch.pvclock_gtod_sync_lock); } -EXPORT_SYMBOL_GPL(kvm_write_tsc); static inline void adjust_tsc_offset_guest(struct kvm_vcpu *vcpu, s64 adjustment) @@ -2821,7 +2776,6 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) } return 0; } -EXPORT_SYMBOL_GPL(kvm_set_msr_common); static int get_msr_mce(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata, bool host) { @@ -3060,7 +3014,6 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) } return 0; } -EXPORT_SYMBOL_GPL(kvm_get_msr_common); /* * Read or write a bunch of msrs. All parameters are kernel addresses. @@ -5299,7 +5252,6 @@ int kvm_read_guest_virt(struct kvm_vcpu *vcpu, return kvm_read_guest_virt_helper(addr, val, bytes, vcpu, access, exception); } -EXPORT_SYMBOL_GPL(kvm_read_guest_virt); static int emulator_read_std(struct x86_emulate_ctxt *ctxt, gva_t addr, void *val, unsigned int bytes, @@ -5384,7 +5336,6 @@ int kvm_write_guest_virt_system(struct kvm_vcpu *vcpu, gva_t addr, void *val, return kvm_write_guest_virt_helper(addr, val, bytes, vcpu, PFERR_WRITE_MASK, exception); } -EXPORT_SYMBOL_GPL(kvm_write_guest_virt_system); int handle_ud(struct kvm_vcpu *vcpu) { @@ -5408,7 +5359,6 @@ int handle_ud(struct kvm_vcpu *vcpu) kvm_queue_exception(vcpu, UD_VECTOR); return 1; } -EXPORT_SYMBOL_GPL(handle_ud); static int vcpu_is_mmio_gpa(struct kvm_vcpu *vcpu, unsigned long gva, gpa_t gpa, bool write) @@ -5848,7 +5798,6 @@ int kvm_emulate_wbinvd(struct kvm_vcpu *vcpu) kvm_emulate_wbinvd_noskip(vcpu); return kvm_skip_emulated_instruction(vcpu); } -EXPORT_SYMBOL_GPL(kvm_emulate_wbinvd); @@ -6249,7 +6198,6 @@ int kvm_inject_realmode_interrupt(struct kvm_vcpu *vcpu, int irq, int inc_eip) return EMULATE_DONE; } -EXPORT_SYMBOL_GPL(kvm_inject_realmode_interrupt); static int handle_emulation_failure(struct kvm_vcpu *vcpu, int emulation_type) { @@ -6461,7 +6409,6 @@ int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu) kvm_vcpu_do_singlestep(vcpu, &r); return r == EMULATE_DONE; } -EXPORT_SYMBOL_GPL(kvm_skip_emulated_instruction); static bool kvm_vcpu_check_breakpoint(struct kvm_vcpu *vcpu, int *r) { @@ -6691,14 +6638,12 @@ int kvm_emulate_instruction(struct kvm_vcpu *vcpu, int emulation_type) { return x86_emulate_instruction(vcpu, 0, emulation_type, NULL, 0); } -EXPORT_SYMBOL_GPL(kvm_emulate_instruction); int kvm_emulate_instruction_from_buffer(struct kvm_vcpu *vcpu, void *insn, int insn_len) { return x86_emulate_instruction(vcpu, 0, 0, insn, insn_len); } -EXPORT_SYMBOL_GPL(kvm_emulate_instruction_from_buffer); static int complete_fast_pio_out_port_0x7e(struct kvm_vcpu *vcpu) { @@ -6799,7 +6744,6 @@ int kvm_fast_pio(struct kvm_vcpu *vcpu, int size, unsigned short port, int in) ret = kvm_fast_pio_out(vcpu, size, port); return ret && kvm_skip_emulated_instruction(vcpu); } -EXPORT_SYMBOL_GPL(kvm_fast_pio); static int kvmclock_cpu_down_prep(unsigned int cpu) { @@ -6986,7 +6930,6 @@ static void kvm_timer_init(void) } DEFINE_PER_CPU(struct kvm_vcpu *, current_vcpu); -EXPORT_PER_CPU_SYMBOL_GPL(current_vcpu); int kvm_is_in_guest(void) { @@ -7190,7 +7133,6 @@ int kvm_vcpu_halt(struct kvm_vcpu *vcpu) return 0; } } -EXPORT_SYMBOL_GPL(kvm_vcpu_halt); int kvm_emulate_halt(struct kvm_vcpu *vcpu) { @@ -7201,7 +7143,6 @@ int kvm_emulate_halt(struct kvm_vcpu *vcpu) */ return kvm_vcpu_halt(vcpu) && ret; } -EXPORT_SYMBOL_GPL(kvm_emulate_halt); #ifdef CONFIG_X86_64 static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr, @@ -7345,7 +7286,6 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) ++vcpu->stat.hypercalls; return kvm_skip_emulated_instruction(vcpu); } -EXPORT_SYMBOL_GPL(kvm_emulate_hypercall); static int emulator_fix_hypercall(struct x86_emulate_ctxt *ctxt) { @@ -7851,13 +7791,11 @@ void kvm_vcpu_reload_apic_access_page(struct kvm_vcpu *vcpu) */ put_page(page); } -EXPORT_SYMBOL_GPL(kvm_vcpu_reload_apic_access_page); void __kvm_request_immediate_exit(struct kvm_vcpu *vcpu) { smp_send_reschedule(vcpu->cpu); } -EXPORT_SYMBOL_GPL(__kvm_request_immediate_exit); /* * Returns 1 to let vcpu_run() continue the guest execution loop without @@ -8533,7 +8471,6 @@ void kvm_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l) *db = cs.db; *l = cs.l; } -EXPORT_SYMBOL_GPL(kvm_get_cs_db_l_bits); static void __get_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) { @@ -8645,7 +8582,6 @@ int kvm_task_switch(struct kvm_vcpu *vcpu, u16 tss_selector, int idt_index, kvm_make_request(KVM_REQ_EVENT, vcpu); return EMULATE_DONE; } -EXPORT_SYMBOL_GPL(kvm_task_switch); static int kvm_valid_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) { @@ -9246,7 +9182,6 @@ bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu) { return vcpu->kvm->arch.bsp_vcpu_id == vcpu->vcpu_id; } -EXPORT_SYMBOL_GPL(kvm_vcpu_is_reset_bsp); bool kvm_vcpu_is_bsp(struct kvm_vcpu *vcpu) { @@ -9254,7 +9189,6 @@ bool kvm_vcpu_is_bsp(struct kvm_vcpu *vcpu) } struct static_key kvm_no_apic_vcpu __read_mostly; -EXPORT_SYMBOL_GPL(kvm_no_apic_vcpu); int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu) { @@ -9476,7 +9410,6 @@ int __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size) return 0; } -EXPORT_SYMBOL_GPL(__x86_set_memory_region); int x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size) { @@ -9488,7 +9421,6 @@ int x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size) return r; } -EXPORT_SYMBOL_GPL(x86_set_memory_region); void kvm_arch_destroy_vm(struct kvm *kvm) { @@ -9805,13 +9737,11 @@ unsigned long kvm_get_linear_rip(struct kvm_vcpu *vcpu) return (u32)(get_segment_base(vcpu, VCPU_SREG_CS) + kvm_rip_read(vcpu)); } -EXPORT_SYMBOL_GPL(kvm_get_linear_rip); bool kvm_is_linear_rip(struct kvm_vcpu *vcpu, unsigned long linear_rip) { return kvm_get_linear_rip(vcpu) == linear_rip; } -EXPORT_SYMBOL_GPL(kvm_is_linear_rip); unsigned long kvm_get_rflags(struct kvm_vcpu *vcpu) { @@ -9822,7 +9752,6 @@ unsigned long kvm_get_rflags(struct kvm_vcpu *vcpu) rflags &= ~X86_EFLAGS_TF; return rflags; } -EXPORT_SYMBOL_GPL(kvm_get_rflags); static void __kvm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags) { @@ -9837,7 +9766,6 @@ void kvm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags) __kvm_set_rflags(vcpu, rflags); kvm_make_request(KVM_REQ_EVENT, vcpu); } -EXPORT_SYMBOL_GPL(kvm_set_rflags); void kvm_arch_async_page_ready(struct kvm_vcpu *vcpu, struct kvm_async_pf *work) { @@ -10044,37 +9972,31 @@ void kvm_arch_start_assignment(struct kvm *kvm) { atomic_inc(&kvm->arch.assigned_device_count); } -EXPORT_SYMBOL_GPL(kvm_arch_start_assignment); void kvm_arch_end_assignment(struct kvm *kvm) { atomic_dec(&kvm->arch.assigned_device_count); } -EXPORT_SYMBOL_GPL(kvm_arch_end_assignment); bool kvm_arch_has_assigned_device(struct kvm *kvm) { return atomic_read(&kvm->arch.assigned_device_count); } -EXPORT_SYMBOL_GPL(kvm_arch_has_assigned_device); void kvm_arch_register_noncoherent_dma(struct kvm *kvm) { atomic_inc(&kvm->arch.noncoherent_dma_count); } -EXPORT_SYMBOL_GPL(kvm_arch_register_noncoherent_dma); void kvm_arch_unregister_noncoherent_dma(struct kvm *kvm) { atomic_dec(&kvm->arch.noncoherent_dma_count); } -EXPORT_SYMBOL_GPL(kvm_arch_unregister_noncoherent_dma); bool kvm_arch_has_noncoherent_dma(struct kvm *kvm) { return atomic_read(&kvm->arch.noncoherent_dma_count); } -EXPORT_SYMBOL_GPL(kvm_arch_has_noncoherent_dma); bool kvm_arch_has_irq_bypass(void) { @@ -10125,32 +10047,9 @@ bool kvm_vector_hashing_enabled(void) { return vector_hashing; } -EXPORT_SYMBOL_GPL(kvm_vector_hashing_enabled); bool kvm_arch_no_poll(struct kvm_vcpu *vcpu) { return (vcpu->arch.msr_kvm_poll_control & 1) == 0; } -EXPORT_SYMBOL_GPL(kvm_arch_no_poll); - - -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_exit); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_fast_mmio); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_inj_virq); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_page_fault); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_msr); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_cr); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmrun); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmexit); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmexit_inject); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_intr_vmexit); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmenter_failed); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_invlpga); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_skinit); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_intercepts); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_write_tsc_offset); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_ple_window_update); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_pml_full); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_pi_irte_update); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_avic_unaccelerated_access); -EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_avic_incomplete_ipi); From patchwork Fri Sep 20 21:25:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154865 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1C16115E6 for ; Fri, 20 Sep 2019 21:25:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DA1BF20B7C for ; Fri, 20 Sep 2019 21:25:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393923AbfITVZr (ORCPT ); Fri, 20 Sep 2019 17:25:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54170 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393808AbfITVZQ (ORCPT ); Fri, 20 Sep 2019 17:25:16 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 38BAA3082E24; Fri, 20 Sep 2019 21:25:16 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 522F95F7C0; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 12/17] KVM: monolithic: remove exports from KVM common code Date: Fri, 20 Sep 2019 17:25:04 -0400 Message-Id: <20190920212509.2578-13-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Fri, 20 Sep 2019 21:25:16 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The exports would be duplicated across kvm-amd and kvm-intel if they're kept and that causes various harmless. The warnings aren't particularly concerning because the two modules can't load at the same time, but it's cleaner to remove the warnings by removing the exports. This commit might break non-x86 archs, but it should be simple to make them monolithic too (if they're not already). In the unlikely case there's a legit reason not to go monolithic in any arch and to keep kvm.ko around, we'll need a way to retain the exports. In which case this commit would need to be reversed and the exports in the kvm common code should then be done only conditionally to a new opt-in per-arch CONFIG option. The following warning remains for now to be able to load the kvmgt driver. These remaining warnings can be handled later. WARNING: arch/x86/kvm/kvm-amd: 'kvm_get_kvm' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko WARNING: arch/x86/kvm/kvm-amd: 'kvm_put_kvm' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko WARNING: arch/x86/kvm/kvm-amd: 'gfn_to_memslot' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko WARNING: arch/x86/kvm/kvm-amd: 'kvm_is_visible_gfn' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko WARNING: arch/x86/kvm/kvm-amd: 'gfn_to_pfn' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko WARNING: arch/x86/kvm/kvm-amd: 'kvm_read_guest' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko WARNING: arch/x86/kvm/kvm-amd: 'kvm_write_guest' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko WARNING: arch/x86/kvm/kvm-amd: 'kvm_slot_page_track_add_page' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko WARNING: arch/x86/kvm/kvm-amd: 'kvm_slot_page_track_remove_page' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko WARNING: arch/x86/kvm/kvm-amd: 'kvm_page_track_register_notifier' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko WARNING: arch/x86/kvm/kvm-amd: 'kvm_page_track_unregister_notifier' exported twice. Previous export was in arch/x86/kvm/kvm-intel.ko a Signed-off-by: Andrea Arcangeli --- virt/kvm/eventfd.c | 1 - virt/kvm/kvm_main.c | 65 --------------------------------------------- 2 files changed, 66 deletions(-) diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c index 67b6fc153e9c..4c1a8abd1458 100644 --- a/virt/kvm/eventfd.c +++ b/virt/kvm/eventfd.c @@ -462,7 +462,6 @@ bool kvm_irq_has_notifier(struct kvm *kvm, unsigned irqchip, unsigned pin) return false; } -EXPORT_SYMBOL_GPL(kvm_irq_has_notifier); void kvm_notify_acked_gsi(struct kvm *kvm, int gsi) { diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 9aa448ea688f..1afbb387001a 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -72,22 +72,18 @@ MODULE_LICENSE("GPL"); /* Architectures should define their poll value according to the halt latency */ unsigned int halt_poll_ns = KVM_HALT_POLL_NS_DEFAULT; module_param(halt_poll_ns, uint, 0644); -EXPORT_SYMBOL_GPL(halt_poll_ns); /* Default doubles per-vcpu halt_poll_ns. */ unsigned int halt_poll_ns_grow = 2; module_param(halt_poll_ns_grow, uint, 0644); -EXPORT_SYMBOL_GPL(halt_poll_ns_grow); /* The start value to grow halt_poll_ns from */ unsigned int halt_poll_ns_grow_start = 10000; /* 10us */ module_param(halt_poll_ns_grow_start, uint, 0644); -EXPORT_SYMBOL_GPL(halt_poll_ns_grow_start); /* Default resets per-vcpu halt_poll_ns . */ unsigned int halt_poll_ns_shrink; module_param(halt_poll_ns_shrink, uint, 0644); -EXPORT_SYMBOL_GPL(halt_poll_ns_shrink); /* * Ordering of locks: @@ -104,12 +100,10 @@ static int kvm_usage_count; static atomic_t hardware_enable_failed; struct kmem_cache *kvm_vcpu_cache; -EXPORT_SYMBOL_GPL(kvm_vcpu_cache); static __read_mostly struct preempt_ops kvm_preempt_ops; struct dentry *kvm_debugfs_dir; -EXPORT_SYMBOL_GPL(kvm_debugfs_dir); static int kvm_debugfs_num_entries; static const struct file_operations *stat_fops_per_vm[]; @@ -133,7 +127,6 @@ static void kvm_io_bus_destroy(struct kvm_io_bus *bus); static void mark_page_dirty_in_slot(struct kvm_memory_slot *memslot, gfn_t gfn); __visible bool kvm_rebooting; -EXPORT_SYMBOL_GPL(kvm_rebooting); static bool largepages_enabled = true; @@ -167,7 +160,6 @@ void vcpu_load(struct kvm_vcpu *vcpu) kvm_arch_vcpu_load(vcpu, cpu); put_cpu(); } -EXPORT_SYMBOL_GPL(vcpu_load); void vcpu_put(struct kvm_vcpu *vcpu) { @@ -176,7 +168,6 @@ void vcpu_put(struct kvm_vcpu *vcpu) preempt_notifier_unregister(&vcpu->preempt_notifier); preempt_enable(); } -EXPORT_SYMBOL_GPL(vcpu_put); /* TODO: merge with kvm_arch_vcpu_should_kick */ static bool kvm_request_needs_ipi(struct kvm_vcpu *vcpu, unsigned req) @@ -280,7 +271,6 @@ void kvm_flush_remote_tlbs(struct kvm *kvm) ++kvm->stat.remote_tlb_flush; cmpxchg(&kvm->tlbs_dirty, dirty_count, 0); } -EXPORT_SYMBOL_GPL(kvm_flush_remote_tlbs); #endif void kvm_reload_remote_mmus(struct kvm *kvm) @@ -326,7 +316,6 @@ int kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id) fail: return r; } -EXPORT_SYMBOL_GPL(kvm_vcpu_init); void kvm_vcpu_uninit(struct kvm_vcpu *vcpu) { @@ -339,7 +328,6 @@ void kvm_vcpu_uninit(struct kvm_vcpu *vcpu) kvm_arch_vcpu_uninit(vcpu); free_page((unsigned long)vcpu->run); } -EXPORT_SYMBOL_GPL(kvm_vcpu_uninit); #if defined(CONFIG_MMU_NOTIFIER) && defined(KVM_ARCH_WANT_MMU_NOTIFIER) static inline struct kvm *mmu_notifier_to_kvm(struct mmu_notifier *mn) @@ -1076,7 +1064,6 @@ int __kvm_set_memory_region(struct kvm *kvm, out: return r; } -EXPORT_SYMBOL_GPL(__kvm_set_memory_region); int kvm_set_memory_region(struct kvm *kvm, const struct kvm_userspace_memory_region *mem) @@ -1088,7 +1075,6 @@ int kvm_set_memory_region(struct kvm *kvm, mutex_unlock(&kvm->slots_lock); return r; } -EXPORT_SYMBOL_GPL(kvm_set_memory_region); static int kvm_vm_ioctl_set_memory_region(struct kvm *kvm, struct kvm_userspace_memory_region *mem) @@ -1130,7 +1116,6 @@ int kvm_get_dirty_log(struct kvm *kvm, *is_dirty = 1; return 0; } -EXPORT_SYMBOL_GPL(kvm_get_dirty_log); #ifdef CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT /** @@ -1216,7 +1201,6 @@ int kvm_get_dirty_log_protect(struct kvm *kvm, return -EFAULT; return 0; } -EXPORT_SYMBOL_GPL(kvm_get_dirty_log_protect); /** * kvm_clear_dirty_log_protect - clear dirty bits in the bitmap @@ -1290,7 +1274,6 @@ int kvm_clear_dirty_log_protect(struct kvm *kvm, return 0; } -EXPORT_SYMBOL_GPL(kvm_clear_dirty_log_protect); #endif bool kvm_largepages_enabled(void) @@ -1302,7 +1285,6 @@ void kvm_disable_largepages(void) { largepages_enabled = false; } -EXPORT_SYMBOL_GPL(kvm_disable_largepages); struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn) { @@ -1382,19 +1364,16 @@ unsigned long gfn_to_hva_memslot(struct kvm_memory_slot *slot, { return gfn_to_hva_many(slot, gfn, NULL); } -EXPORT_SYMBOL_GPL(gfn_to_hva_memslot); unsigned long gfn_to_hva(struct kvm *kvm, gfn_t gfn) { return gfn_to_hva_many(gfn_to_memslot(kvm, gfn), gfn, NULL); } -EXPORT_SYMBOL_GPL(gfn_to_hva); unsigned long kvm_vcpu_gfn_to_hva(struct kvm_vcpu *vcpu, gfn_t gfn) { return gfn_to_hva_many(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn, NULL); } -EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_hva); /* * Return the hva of a @gfn and the R/W attribute if possible. @@ -1656,7 +1635,6 @@ kvm_pfn_t __gfn_to_pfn_memslot(struct kvm_memory_slot *slot, gfn_t gfn, return hva_to_pfn(addr, atomic, async, write_fault, writable); } -EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) @@ -1664,31 +1642,26 @@ kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, NULL, write_fault, writable); } -EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); kvm_pfn_t gfn_to_pfn_memslot(struct kvm_memory_slot *slot, gfn_t gfn) { return __gfn_to_pfn_memslot(slot, gfn, false, NULL, true, NULL); } -EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); kvm_pfn_t gfn_to_pfn_memslot_atomic(struct kvm_memory_slot *slot, gfn_t gfn) { return __gfn_to_pfn_memslot(slot, gfn, true, NULL, true, NULL); } -EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot_atomic); kvm_pfn_t gfn_to_pfn_atomic(struct kvm *kvm, gfn_t gfn) { return gfn_to_pfn_memslot_atomic(gfn_to_memslot(kvm, gfn), gfn); } -EXPORT_SYMBOL_GPL(gfn_to_pfn_atomic); kvm_pfn_t kvm_vcpu_gfn_to_pfn_atomic(struct kvm_vcpu *vcpu, gfn_t gfn) { return gfn_to_pfn_memslot_atomic(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn); } -EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn_atomic); kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn) { @@ -1700,7 +1673,6 @@ kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn) { return gfn_to_pfn_memslot(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn); } -EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn); int gfn_to_page_many_atomic(struct kvm_memory_slot *slot, gfn_t gfn, struct page **pages, int nr_pages) @@ -1717,7 +1689,6 @@ int gfn_to_page_many_atomic(struct kvm_memory_slot *slot, gfn_t gfn, return __get_user_pages_fast(addr, nr_pages, 1, pages); } -EXPORT_SYMBOL_GPL(gfn_to_page_many_atomic); static struct page *kvm_pfn_to_page(kvm_pfn_t pfn) { @@ -1740,7 +1711,6 @@ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) return kvm_pfn_to_page(pfn); } -EXPORT_SYMBOL_GPL(gfn_to_page); static int __kvm_map_gfn(struct kvm_memory_slot *slot, gfn_t gfn, struct kvm_host_map *map) @@ -1780,7 +1750,6 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) { return __kvm_map_gfn(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn, map); } -EXPORT_SYMBOL_GPL(kvm_vcpu_map); void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) @@ -1808,7 +1777,6 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, map->hva = NULL; map->page = NULL; } -EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); struct page *kvm_vcpu_gfn_to_page(struct kvm_vcpu *vcpu, gfn_t gfn) { @@ -1818,7 +1786,6 @@ struct page *kvm_vcpu_gfn_to_page(struct kvm_vcpu *vcpu, gfn_t gfn) return kvm_pfn_to_page(pfn); } -EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_page); void kvm_release_page_clean(struct page *page) { @@ -1826,14 +1793,12 @@ void kvm_release_page_clean(struct page *page) kvm_release_pfn_clean(page_to_pfn(page)); } -EXPORT_SYMBOL_GPL(kvm_release_page_clean); void kvm_release_pfn_clean(kvm_pfn_t pfn) { if (!is_error_noslot_pfn(pfn) && !kvm_is_reserved_pfn(pfn)) put_page(pfn_to_page(pfn)); } -EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); void kvm_release_page_dirty(struct page *page) { @@ -1841,14 +1806,12 @@ void kvm_release_page_dirty(struct page *page) kvm_release_pfn_dirty(page_to_pfn(page)); } -EXPORT_SYMBOL_GPL(kvm_release_page_dirty); void kvm_release_pfn_dirty(kvm_pfn_t pfn) { kvm_set_pfn_dirty(pfn); kvm_release_pfn_clean(pfn); } -EXPORT_SYMBOL_GPL(kvm_release_pfn_dirty); void kvm_set_pfn_dirty(kvm_pfn_t pfn) { @@ -1858,21 +1821,18 @@ void kvm_set_pfn_dirty(kvm_pfn_t pfn) SetPageDirty(page); } } -EXPORT_SYMBOL_GPL(kvm_set_pfn_dirty); void kvm_set_pfn_accessed(kvm_pfn_t pfn) { if (!kvm_is_reserved_pfn(pfn)) mark_page_accessed(pfn_to_page(pfn)); } -EXPORT_SYMBOL_GPL(kvm_set_pfn_accessed); void kvm_get_pfn(kvm_pfn_t pfn) { if (!kvm_is_reserved_pfn(pfn)) get_page(pfn_to_page(pfn)); } -EXPORT_SYMBOL_GPL(kvm_get_pfn); static int next_segment(unsigned long len, int offset) { @@ -1904,7 +1864,6 @@ int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset, return __kvm_read_guest_page(slot, gfn, data, offset, len); } -EXPORT_SYMBOL_GPL(kvm_read_guest_page); int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data, int offset, int len) @@ -1913,7 +1872,6 @@ int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data, return __kvm_read_guest_page(slot, gfn, data, offset, len); } -EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_page); int kvm_read_guest(struct kvm *kvm, gpa_t gpa, void *data, unsigned long len) { @@ -1953,7 +1911,6 @@ int kvm_vcpu_read_guest(struct kvm_vcpu *vcpu, gpa_t gpa, void *data, unsigned l } return 0; } -EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest); static int __kvm_read_guest_atomic(struct kvm_memory_slot *slot, gfn_t gfn, void *data, int offset, unsigned long len) @@ -1981,7 +1938,6 @@ int kvm_read_guest_atomic(struct kvm *kvm, gpa_t gpa, void *data, return __kvm_read_guest_atomic(slot, gfn, data, offset, len); } -EXPORT_SYMBOL_GPL(kvm_read_guest_atomic); int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu, gpa_t gpa, void *data, unsigned long len) @@ -1992,7 +1948,6 @@ int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu, gpa_t gpa, return __kvm_read_guest_atomic(slot, gfn, data, offset, len); } -EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_atomic); static int __kvm_write_guest_page(struct kvm_memory_slot *memslot, gfn_t gfn, const void *data, int offset, int len) @@ -2017,7 +1972,6 @@ int kvm_write_guest_page(struct kvm *kvm, gfn_t gfn, return __kvm_write_guest_page(slot, gfn, data, offset, len); } -EXPORT_SYMBOL_GPL(kvm_write_guest_page); int kvm_vcpu_write_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, const void *data, int offset, int len) @@ -2026,7 +1980,6 @@ int kvm_vcpu_write_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, return __kvm_write_guest_page(slot, gfn, data, offset, len); } -EXPORT_SYMBOL_GPL(kvm_vcpu_write_guest_page); int kvm_write_guest(struct kvm *kvm, gpa_t gpa, const void *data, unsigned long len) @@ -2068,7 +2021,6 @@ int kvm_vcpu_write_guest(struct kvm_vcpu *vcpu, gpa_t gpa, const void *data, } return 0; } -EXPORT_SYMBOL_GPL(kvm_vcpu_write_guest); static int __kvm_gfn_to_hva_cache_init(struct kvm_memslots *slots, struct gfn_to_hva_cache *ghc, @@ -2114,7 +2066,6 @@ int kvm_gfn_to_hva_cache_init(struct kvm *kvm, struct gfn_to_hva_cache *ghc, struct kvm_memslots *slots = kvm_memslots(kvm); return __kvm_gfn_to_hva_cache_init(slots, ghc, gpa, len); } -EXPORT_SYMBOL_GPL(kvm_gfn_to_hva_cache_init); int kvm_write_guest_offset_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, void *data, unsigned int offset, @@ -2142,14 +2093,12 @@ int kvm_write_guest_offset_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, return 0; } -EXPORT_SYMBOL_GPL(kvm_write_guest_offset_cached); int kvm_write_guest_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, void *data, unsigned long len) { return kvm_write_guest_offset_cached(kvm, ghc, data, 0, len); } -EXPORT_SYMBOL_GPL(kvm_write_guest_cached); int kvm_read_guest_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, void *data, unsigned long len) @@ -2174,7 +2123,6 @@ int kvm_read_guest_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, return 0; } -EXPORT_SYMBOL_GPL(kvm_read_guest_cached); int kvm_clear_guest_page(struct kvm *kvm, gfn_t gfn, int offset, int len) { @@ -2182,7 +2130,6 @@ int kvm_clear_guest_page(struct kvm *kvm, gfn_t gfn, int offset, int len) return kvm_write_guest_page(kvm, gfn, zero_page, offset, len); } -EXPORT_SYMBOL_GPL(kvm_clear_guest_page); int kvm_clear_guest(struct kvm *kvm, gpa_t gpa, unsigned long len) { @@ -2201,7 +2148,6 @@ int kvm_clear_guest(struct kvm *kvm, gpa_t gpa, unsigned long len) } return 0; } -EXPORT_SYMBOL_GPL(kvm_clear_guest); static void mark_page_dirty_in_slot(struct kvm_memory_slot *memslot, gfn_t gfn) @@ -2220,7 +2166,6 @@ void mark_page_dirty(struct kvm *kvm, gfn_t gfn) memslot = gfn_to_memslot(kvm, gfn); mark_page_dirty_in_slot(memslot, gfn); } -EXPORT_SYMBOL_GPL(mark_page_dirty); void kvm_vcpu_mark_page_dirty(struct kvm_vcpu *vcpu, gfn_t gfn) { @@ -2229,7 +2174,6 @@ void kvm_vcpu_mark_page_dirty(struct kvm_vcpu *vcpu, gfn_t gfn) memslot = kvm_vcpu_gfn_to_memslot(vcpu, gfn); mark_page_dirty_in_slot(memslot, gfn); } -EXPORT_SYMBOL_GPL(kvm_vcpu_mark_page_dirty); void kvm_sigset_activate(struct kvm_vcpu *vcpu) { @@ -2377,7 +2321,6 @@ void kvm_vcpu_block(struct kvm_vcpu *vcpu) trace_kvm_vcpu_wakeup(block_ns, waited, vcpu_valid_wakeup(vcpu)); kvm_arch_vcpu_block_finish(vcpu); } -EXPORT_SYMBOL_GPL(kvm_vcpu_block); bool kvm_vcpu_wake_up(struct kvm_vcpu *vcpu) { @@ -2393,7 +2336,6 @@ bool kvm_vcpu_wake_up(struct kvm_vcpu *vcpu) return false; } -EXPORT_SYMBOL_GPL(kvm_vcpu_wake_up); #ifndef CONFIG_S390 /* @@ -2413,7 +2355,6 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu) smp_send_reschedule(cpu); put_cpu(); } -EXPORT_SYMBOL_GPL(kvm_vcpu_kick); #endif /* !CONFIG_S390 */ int kvm_vcpu_yield_to(struct kvm_vcpu *target) @@ -2434,7 +2375,6 @@ int kvm_vcpu_yield_to(struct kvm_vcpu *target) return ret; } -EXPORT_SYMBOL_GPL(kvm_vcpu_yield_to); /* * Helper that checks whether a VCPU is eligible for directed yield. @@ -2551,7 +2491,6 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me, bool yield_to_kernel_mode) /* Ensure vcpu is not eligible during next spinloop */ kvm_vcpu_set_dy_eligible(me, false); } -EXPORT_SYMBOL_GPL(kvm_vcpu_on_spin); static vm_fault_t kvm_vcpu_fault(struct vm_fault *vmf) { @@ -3735,7 +3674,6 @@ int kvm_io_bus_write(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr, r = __kvm_io_bus_write(vcpu, bus, &range, val); return r < 0 ? r : 0; } -EXPORT_SYMBOL_GPL(kvm_io_bus_write); /* kvm_io_bus_write_cookie - called under kvm->slots_lock */ int kvm_io_bus_write_cookie(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, @@ -3912,7 +3850,6 @@ struct kvm_io_device *kvm_io_bus_get_dev(struct kvm *kvm, enum kvm_bus bus_idx, return iodev; } -EXPORT_SYMBOL_GPL(kvm_io_bus_get_dev); static int kvm_debugfs_open(struct inode *inode, struct file *file, int (*get)(void *, u64 *), int (*set)(void *, u64), @@ -4341,7 +4278,6 @@ __init int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align, out_fail: return r; } -EXPORT_SYMBOL_GPL(kvm_init); void kvm_exit(void) { @@ -4359,4 +4295,3 @@ void kvm_exit(void) free_cpumask_var(cpus_hardware_enabled); kvm_vfio_ops_exit(); } -EXPORT_SYMBOL_GPL(kvm_exit); From patchwork Fri Sep 20 21:25:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154873 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2F5781599 for ; Fri, 20 Sep 2019 21:26:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 17AB2208C3 for ; Fri, 20 Sep 2019 21:26:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393972AbfITV0D (ORCPT ); Fri, 20 Sep 2019 17:26:03 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57986 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392508AbfITVZO (ORCPT ); Fri, 20 Sep 2019 17:25:14 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 902C58980E5; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5DA4C1001B2D; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 13/17] KVM: monolithic: x86: drop the kvm_pmu_ops structure Date: Fri, 20 Sep 2019 17:25:05 -0400 Message-Id: <20190920212509.2578-14-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.67]); Fri, 20 Sep 2019 21:25:13 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Cleanup after this was finally left fully unused. Signed-off-by: Andrea Arcangeli --- arch/x86/include/asm/kvm_host.h | 3 --- arch/x86/kvm/pmu.h | 19 ------------------- arch/x86/kvm/pmu_amd.c | 15 --------------- arch/x86/kvm/svm.c | 1 - arch/x86/kvm/vmx/pmu_intel.c | 15 --------------- arch/x86/kvm/vmx/vmx.c | 2 -- 6 files changed, 55 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index bd5f4f900288..000d4e5a5664 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1160,9 +1160,6 @@ struct kvm_x86_ops { gfn_t offset, unsigned long mask); int (*write_log_dirty)(struct kvm_vcpu *vcpu); - /* pmu operations of sub-arch */ - const struct kvm_pmu_ops *pmu_ops; - /* * Architecture specific hooks for vCPU blocking due to * HLT instruction. diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index 09e80e8ee21a..513366c8b794 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -21,23 +21,6 @@ struct kvm_event_hw_type_mapping { #include "pmu_ops.h" -struct kvm_pmu_ops { - unsigned (*find_arch_event)(struct kvm_pmu *pmu, u8 event_select, - u8 unit_mask); - unsigned (*find_fixed_event)(int idx); - bool (*pmc_is_enabled)(struct kvm_pmc *pmc); - struct kvm_pmc *(*pmc_idx_to_pmc)(struct kvm_pmu *pmu, int pmc_idx); - struct kvm_pmc *(*msr_idx_to_pmc)(struct kvm_vcpu *vcpu, unsigned idx, - u64 *mask); - int (*is_valid_msr_idx)(struct kvm_vcpu *vcpu, unsigned idx); - bool (*is_valid_msr)(struct kvm_vcpu *vcpu, u32 msr); - int (*get_msr)(struct kvm_vcpu *vcpu, u32 msr, u64 *data); - int (*set_msr)(struct kvm_vcpu *vcpu, struct msr_data *msr_info); - void (*refresh)(struct kvm_vcpu *vcpu); - void (*init)(struct kvm_vcpu *vcpu); - void (*reset)(struct kvm_vcpu *vcpu); -}; - static inline u64 pmc_bitmask(struct kvm_pmc *pmc) { struct kvm_pmu *pmu = pmc_to_pmu(pmc); @@ -124,6 +107,4 @@ int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp); bool is_vmware_backdoor_pmc(u32 pmc_idx); -extern struct kvm_pmu_ops intel_pmu_ops; -extern struct kvm_pmu_ops amd_pmu_ops; #endif /* __KVM_X86_PMU_H */ diff --git a/arch/x86/kvm/pmu_amd.c b/arch/x86/kvm/pmu_amd.c index 12d1fa3ba35a..0e01b48b3b0b 100644 --- a/arch/x86/kvm/pmu_amd.c +++ b/arch/x86/kvm/pmu_amd.c @@ -302,18 +302,3 @@ static void amd_pmu_reset(struct kvm_vcpu *vcpu) } #include "pmu_amd_ops.c" - -struct kvm_pmu_ops amd_pmu_ops = { - .find_arch_event = amd_find_arch_event, - .find_fixed_event = amd_find_fixed_event, - .pmc_is_enabled = amd_pmc_is_enabled, - .pmc_idx_to_pmc = amd_pmc_idx_to_pmc, - .msr_idx_to_pmc = amd_msr_idx_to_pmc, - .is_valid_msr_idx = amd_is_valid_msr_idx, - .is_valid_msr = amd_is_valid_msr, - .get_msr = amd_pmu_get_msr, - .set_msr = amd_pmu_set_msr, - .refresh = amd_pmu_refresh, - .init = amd_pmu_init, - .reset = amd_pmu_reset, -}; diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index 5a48beb58083..ca9fd1762519 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -7298,7 +7298,6 @@ static struct kvm_x86_ops svm_x86_ops __ro_after_init = { .sched_in = svm_sched_in, - .pmu_ops = &amd_pmu_ops, .deliver_posted_interrupt = svm_deliver_avic_intr, .dy_apicv_has_pending_interrupt = svm_dy_apicv_has_pending_interrupt, .update_pi_irte = svm_update_pi_irte, diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index bfa765842772..e7c1253f47a2 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -359,18 +359,3 @@ static void intel_pmu_reset(struct kvm_vcpu *vcpu) } #include "pmu_intel_ops.c" - -struct kvm_pmu_ops intel_pmu_ops = { - .find_arch_event = intel_find_arch_event, - .find_fixed_event = intel_find_fixed_event, - .pmc_is_enabled = intel_pmc_is_enabled, - .pmc_idx_to_pmc = intel_pmc_idx_to_pmc, - .msr_idx_to_pmc = intel_msr_idx_to_pmc, - .is_valid_msr_idx = intel_is_valid_msr_idx, - .is_valid_msr = intel_is_valid_msr, - .get_msr = intel_pmu_get_msr, - .set_msr = intel_pmu_set_msr, - .refresh = intel_pmu_refresh, - .init = intel_pmu_init, - .reset = intel_pmu_reset, -}; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 09e6a477e06f..ff46008dc514 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7779,8 +7779,6 @@ static struct kvm_x86_ops vmx_x86_ops __ro_after_init = { .pre_block = vmx_pre_block, .post_block = vmx_post_block, - .pmu_ops = &intel_pmu_ops, - .update_pi_irte = vmx_update_pi_irte, #ifdef CONFIG_X86_64 From patchwork Fri Sep 20 21:25:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154883 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BDCA315E6 for ; Fri, 20 Sep 2019 21:26:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9C96E2086A for ; Fri, 20 Sep 2019 21:26:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393996AbfITV0Q (ORCPT ); Fri, 20 Sep 2019 17:26:16 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53098 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392265AbfITVZN (ORCPT ); Fri, 20 Sep 2019 17:25:13 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6A1707F743; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 49FD210013D9; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 14/17] KVM: monolithic: x86: inline more exit handlers in vmx.c Date: Fri, 20 Sep 2019 17:25:06 -0400 Message-Id: <20190920212509.2578-15-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.71]); Fri, 20 Sep 2019 21:25:13 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org They can be called directly more efficiently, so we can as well mark some of them inline in case gcc doesn't decide to inline them. Signed-off-by: Andrea Arcangeli --- arch/x86/kvm/vmx/vmx.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index ff46008dc514..a6e597025011 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -4588,7 +4588,7 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu) return 0; } -static int handle_external_interrupt(struct kvm_vcpu *vcpu) +static __always_inline int handle_external_interrupt(struct kvm_vcpu *vcpu) { ++vcpu->stat.irq_exits; return 1; @@ -4860,7 +4860,7 @@ static void vmx_set_dr7(struct kvm_vcpu *vcpu, unsigned long val) vmcs_writel(GUEST_DR7, val); } -static int handle_cpuid(struct kvm_vcpu *vcpu) +static __always_inline int handle_cpuid(struct kvm_vcpu *vcpu) { return kvm_emulate_cpuid(vcpu); } @@ -4891,7 +4891,7 @@ static int handle_interrupt_window(struct kvm_vcpu *vcpu) return 1; } -static int handle_halt(struct kvm_vcpu *vcpu) +static __always_inline int handle_halt(struct kvm_vcpu *vcpu) { return kvm_emulate_halt(vcpu); } From patchwork Fri Sep 20 21:25:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154859 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C378F1599 for ; Fri, 20 Sep 2019 21:25:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A1A1620820 for ; Fri, 20 Sep 2019 21:25:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393830AbfITVZS (ORCPT ); Fri, 20 Sep 2019 17:25:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50848 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393804AbfITVZQ (ORCPT ); Fri, 20 Sep 2019 17:25:16 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 075EF3082A6C; Fri, 20 Sep 2019 21:25:16 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DC83C5F7D5; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 15/17] KVM: retpolines: x86: eliminate retpoline from vmx.c exit handlers Date: Fri, 20 Sep 2019 17:25:07 -0400 Message-Id: <20190920212509.2578-16-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Fri, 20 Sep 2019 21:25:16 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org It's enough to check the exit value and issue a direct call to avoid the retpoline for all the common vmexit reasons. Signed-off-by: Andrea Arcangeli --- arch/x86/kvm/vmx/vmx.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index a6e597025011..9aa73e216df2 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -5866,9 +5866,29 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu) } if (exit_reason < kvm_vmx_max_exit_handlers - && kvm_vmx_exit_handlers[exit_reason]) + && kvm_vmx_exit_handlers[exit_reason]) { +#ifdef CONFIG_RETPOLINE + if (exit_reason == EXIT_REASON_MSR_WRITE) + return handle_wrmsr(vcpu); + else if (exit_reason == EXIT_REASON_PREEMPTION_TIMER) + return handle_preemption_timer(vcpu); + else if (exit_reason == EXIT_REASON_PENDING_INTERRUPT) + return handle_interrupt_window(vcpu); + else if (exit_reason == EXIT_REASON_EXTERNAL_INTERRUPT) + return handle_external_interrupt(vcpu); + else if (exit_reason == EXIT_REASON_HLT) + return handle_halt(vcpu); + else if (exit_reason == EXIT_REASON_PAUSE_INSTRUCTION) + return handle_pause(vcpu); + else if (exit_reason == EXIT_REASON_MSR_READ) + return handle_rdmsr(vcpu); + else if (exit_reason == EXIT_REASON_CPUID) + return handle_cpuid(vcpu); + else if (exit_reason == EXIT_REASON_EPT_MISCONFIG) + return handle_ept_misconfig(vcpu); +#endif return kvm_vmx_exit_handlers[exit_reason](vcpu); - else { + } else { vcpu_unimpl(vcpu, "vmx: unexpected exit reason 0x%x\n", exit_reason); dump_vmcs(); From patchwork Fri Sep 20 21:25:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154857 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 578B51599 for ; Fri, 20 Sep 2019 21:25:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3730921882 for ; Fri, 20 Sep 2019 21:25:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393812AbfITVZT (ORCPT ); Fri, 20 Sep 2019 17:25:19 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46018 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393813AbfITVZR (ORCPT ); Fri, 20 Sep 2019 17:25:17 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B706518CB8F9; Fri, 20 Sep 2019 21:25:16 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DE203600C6; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 16/17] KVM: retpolines: x86: eliminate retpoline from svm.c exit handlers Date: Fri, 20 Sep 2019 17:25:08 -0400 Message-Id: <20190920212509.2578-17-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.63]); Fri, 20 Sep 2019 21:25:16 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org It's enough to check the exit value and issue a direct call to avoid the retpoline for all the common vmexit reasons. After this commit is applied, here the most common retpolines executed under a high resolution timer workload in the guest on a SVM host: [..] @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 ktime_get_update_offsets_now+70 hrtimer_interrupt+131 smp_apic_timer_interrupt+106 apic_timer_interrupt+15 start_sw_timer+359 restart_apic_timer+85 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 1940 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_r12+33 force_qs_rnp+217 rcu_gp_kthread+1270 kthread+268 ret_from_fork+34 ]: 4644 @[]: 25095 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 lapic_next_event+28 clockevents_program_event+148 hrtimer_start_range_ns+528 start_sw_timer+356 restart_apic_timer+85 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 41474 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 clockevents_program_event+148 hrtimer_start_range_ns+528 start_sw_timer+356 restart_apic_timer+85 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 41474 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 ktime_get+58 clockevents_program_event+84 hrtimer_start_range_ns+528 start_sw_timer+356 restart_apic_timer+85 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 41887 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 lapic_next_event+28 clockevents_program_event+148 hrtimer_try_to_cancel+168 hrtimer_cancel+21 kvm_set_lapic_tscdeadline_msr+43 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 42723 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 clockevents_program_event+148 hrtimer_try_to_cancel+168 hrtimer_cancel+21 kvm_set_lapic_tscdeadline_msr+43 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 42766 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 ktime_get+58 clockevents_program_event+84 hrtimer_try_to_cancel+168 hrtimer_cancel+21 kvm_set_lapic_tscdeadline_msr+43 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 42848 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 ktime_get+58 start_sw_timer+279 restart_apic_timer+85 kvm_set_msr_common+1497 msr_interception+142 vcpu_enter_guest+684 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 499845 @total: 1780243 SVM has no TSC based programmable preemption timer so it is invoking ktime_get() frequently. Signed-off-by: Andrea Arcangeli --- arch/x86/kvm/svm.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index ca9fd1762519..7324243b0a82 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -4991,6 +4991,20 @@ static int handle_exit(struct kvm_vcpu *vcpu) return 0; } +#ifdef CONFIG_RETPOLINE + if (exit_code == SVM_EXIT_MSR) + return msr_interception(svm); + else if (exit_code == SVM_EXIT_VINTR) + return interrupt_window_interception(svm); + else if (exit_code == SVM_EXIT_INTR) + return intr_interception(svm); + else if (exit_code == SVM_EXIT_HLT) + return halt_interception(svm); + else if (exit_code == SVM_EXIT_NPF) + return npf_interception(svm); + else if (exit_code == SVM_EXIT_CPUID) + return cpuid_interception(svm); +#endif return svm_exit_handlers[exit_code](svm); } From patchwork Fri Sep 20 21:25:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 11154875 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CCE671599 for ; Fri, 20 Sep 2019 21:26:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AA3B420820 for ; Fri, 20 Sep 2019 21:26:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393915AbfITV0C (ORCPT ); Fri, 20 Sep 2019 17:26:02 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57988 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393779AbfITVZO (ORCPT ); Fri, 20 Sep 2019 17:25:14 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 10B208980EA; Fri, 20 Sep 2019 21:25:14 +0000 (UTC) Received: from mail (ovpn-120-159.rdu2.redhat.com [10.10.120.159]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E76295D6B2; Fri, 20 Sep 2019 21:25:13 +0000 (UTC) From: Andrea Arcangeli To: Paolo Bonzini Cc: Vitaly Kuznetsov , "Dr. David Alan Gilbert" , Marcelo Tosatti , Peter Xu , kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 17/17] x86: retpolines: eliminate retpoline from msr event handlers Date: Fri, 20 Sep 2019 17:25:09 -0400 Message-Id: <20190920212509.2578-18-aarcange@redhat.com> In-Reply-To: <20190920212509.2578-1-aarcange@redhat.com> References: <20190920212509.2578-1-aarcange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.67]); Fri, 20 Sep 2019 21:25:14 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org It's enough to check the value and issue the direct call. After this commit is applied, here the most common retpolines executed under a high resolution timer workload in the guest on a VMX host: [..] @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 267 @[]: 2256 @[ trace_retpoline+1 __trace_retpoline+30 __x86_indirect_thunk_rax+33 __kvm_wait_lapic_expire+284 vmx_vcpu_run.part.97+1091 vcpu_enter_guest+377 kvm_arch_vcpu_ioctl_run+261 kvm_vcpu_ioctl+559 do_vfs_ioctl+164 ksys_ioctl+96 __x64_sys_ioctl+22 do_syscall_64+89 entry_SYSCALL_64_after_hwframe+68 ]: 2390 @[]: 33410 @total: 315707 Note the highest hit above is __delay so probably not worth optimizing even if it would be more frequent than 2k hits per sec. Signed-off-by: Andrea Arcangeli --- arch/x86/events/intel/core.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index 27ee47a7be66..65b383d5e062 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -3323,8 +3323,19 @@ static int intel_pmu_hw_config(struct perf_event *event) return 0; } +#ifdef CONFIG_RETPOLINE +static struct perf_guest_switch_msr *core_guest_get_msrs(int *nr); +static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr); +#endif + struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr) { +#ifdef CONFIG_RETPOLINE + if (x86_pmu.guest_get_msrs == intel_guest_get_msrs) + return intel_guest_get_msrs(nr); + else if (x86_pmu.guest_get_msrs == core_guest_get_msrs) + return core_guest_get_msrs(nr); +#endif if (x86_pmu.guest_get_msrs) return x86_pmu.guest_get_msrs(nr); *nr = 0;