From patchwork Mon Jul 27 23:17:39 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Rutherford X-Patchwork-Id: 6877401 Return-Path: X-Original-To: patchwork-kvm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 77620C05AC for ; Mon, 27 Jul 2015 23:18:24 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 2100C20503 for ; Mon, 27 Jul 2015 23:18:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AB34E20555 for ; Mon, 27 Jul 2015 23:18:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754858AbbG0XSO (ORCPT ); Mon, 27 Jul 2015 19:18:14 -0400 Received: from mail-pd0-f178.google.com ([209.85.192.178]:35696 "EHLO mail-pd0-f178.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753968AbbG0XSJ (ORCPT ); Mon, 27 Jul 2015 19:18:09 -0400 Received: by pdrg1 with SMTP id g1so59409784pdr.2 for ; Mon, 27 Jul 2015 16:18:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=from:to:subject:date:message-id; bh=cYvJ+QVK9hK9HNUqF3MM9gOattpPIA5JKszxbffxDas=; b=VmYm9Gexv9A4tsfCHlmbvi1jS68XofS9fmAVMxSZ7enV/59/rKsIh0heDHs0lXvP0A PQCuWTaOidrK1zygGAJvGtNbQ94agz/8TLVyomcgZwTWIS9CjPtK1wO3i/Oei2ubgXbS fOhYmkW+1VNT0kaf1n9x6fEZFIvOLgFhmSMezPmAtKRGUWhUX+2mXz6rOKOqLh3/nB+6 YiL/8l+ktM1yLsCAX3vgvsjQieveo/cUblp2f1/DQMKMaF1fHIcfLzE85ylLPXXrOJfR fxyKQ6BpQrDcRkIA+gRizzUTBPt6t5MC4CGRzVh4+xAjDDk36RFemjJbRP+s3NXbIjxN 5rzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id; bh=cYvJ+QVK9hK9HNUqF3MM9gOattpPIA5JKszxbffxDas=; b=EUIx+2LWP73opKBkilPKdJOwPK3Cne3MHhvPnfE383AI5chnUH0q1CjUcERBt3x5ji 8TbApJs0+ocUP/ozqU3B9nIa3GQoDl0sZaL0hx0iZvimIuBWJKJ5S92DTHpZXRnD7EvJ Tn4k6S24AcEyMw7pYQV2iEXJZtY1SA9h5WY5h5OyZH9YC7qansZ645MdtDNJUo4jRtQb 8dhHfiw6iYD8E6B1QDPsWZHDovpdfS4X5eieypl5VpEuYXVPPKWePZW5k7+jiCLCCzfb 3IGjl2nrqeSNhlyA+pY+5guVFSnm45P1KXRzEWortc344/S+FF393x/KHVvz0zc04Xch dcwA== X-Gm-Message-State: ALoCoQn0BpFEs9v8SafBSe+m1SfKZiFoAMYrYmhY3cGVOsKGJh55OUT9ZvOLNUirStWzYEruiM2B X-Received: by 10.70.88.176 with SMTP id bh16mr73578981pdb.165.1438039089113; Mon, 27 Jul 2015 16:18:09 -0700 (PDT) Received: from entropic.kir.corp.google.com ([172.31.8.128]) by smtp.gmail.com with ESMTPSA id fi7sm31456809pdb.25.2015.07.27.16.18.07 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 27 Jul 2015 16:18:08 -0700 (PDT) From: Steve Rutherford To: kvm@vger.kernel.org Subject: [PATCH v5 1/4] KVM: x86: Split the APIC from the rest of IRQCHIP. Date: Mon, 27 Jul 2015 16:17:39 -0700 Message-Id: <1438039062-3168-1-git-send-email-srutherford@google.com> X-Mailer: git-send-email 2.5.0.rc2.392.g76e840b Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Spam-Status: No, score=-8.1 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP First patch in a series which enables the relocation of the PIC/IOAPIC to userspace. Adds capability KVM_CAP_SPLIT_IRQCHIP; KVM_CAP_SPLIT_IRQCHIP enables the construction of LAPICs without the rest of the irqchip. Compile tested for x86. Signed-off-by: Steve Rutherford Suggested-by: Andrew Honig --- Documentation/virtual/kvm/api.txt | 15 +++++++++++++++ arch/powerpc/kvm/irq.h | 1 - arch/s390/kvm/irq.h | 1 - arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/i8254.c | 5 ++++- arch/x86/kvm/ioapic.h | 9 +++++++++ arch/x86/kvm/irq.h | 6 ++++++ arch/x86/kvm/irq_comm.c | 9 ++++++++- arch/x86/kvm/lapic.c | 9 ++++++--- arch/x86/kvm/vmx.c | 4 ++-- arch/x86/kvm/x86.c | 23 +++++++++++++++++++++-- include/kvm/arm_vgic.h | 1 + include/linux/kvm_host.h | 1 + include/uapi/linux/kvm.h | 1 + 14 files changed, 76 insertions(+), 11 deletions(-) diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt index a4ebcb7..b655024 100644 --- a/Documentation/virtual/kvm/api.txt +++ b/Documentation/virtual/kvm/api.txt @@ -3620,6 +3620,21 @@ struct { KVM handlers should exit to userspace with rc = -EREMOTE. +7.5 KVM_SPLIT_IRQCHIP + +Architectures: x86 +Parameters: None +Returns: 0 on success, -1 on error + +Create a local apic for each processor in the kernel. With this capability +enabled, the userspace VMM is expected to emulate the IOAPIC and PIC. + +This supersedes KVM_CREATE_IRQCHIP, creating only local APICs, but no in kernel +IOAPIC or PIC. This also enables in kernel routing of interrupt requests. + +Fails if VCPU has already been created, or if the irqchip is already in the +kernel (i.e. KVM_CREATE_IRQCHIP has already been called). + 8. Other capabilities. ---------------------- diff --git a/arch/powerpc/kvm/irq.h b/arch/powerpc/kvm/irq.h index 5a9a10b..772fa8c 100644 --- a/arch/powerpc/kvm/irq.h +++ b/arch/powerpc/kvm/irq.h @@ -16,5 +16,4 @@ static inline int irqchip_in_kernel(struct kvm *kvm) smp_rmb(); return ret; } - #endif diff --git a/arch/s390/kvm/irq.h b/arch/s390/kvm/irq.h index d98e415..9a21a86 100644 --- a/arch/s390/kvm/irq.h +++ b/arch/s390/kvm/irq.h @@ -18,5 +18,4 @@ static inline int irqchip_in_kernel(struct kvm *kvm) { return 1; } - #endif diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index fa32b53..18a110b 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -669,6 +669,8 @@ struct kvm_arch { bool boot_vcpu_runs_old_kvmclock; u64 disabled_quirks; + + bool irqchip_split; }; struct kvm_vm_stat { diff --git a/arch/x86/kvm/i8254.c b/arch/x86/kvm/i8254.c index f90952f..5708850 100644 --- a/arch/x86/kvm/i8254.c +++ b/arch/x86/kvm/i8254.c @@ -35,6 +35,7 @@ #include #include +#include "ioapic.h" #include "irq.h" #include "i8254.h" #include "x86.h" @@ -333,7 +334,9 @@ static void create_pit_timer(struct kvm *kvm, u32 val, int is_period) struct kvm_kpit_state *ps = &kvm->arch.vpit->pit_state; s64 interval; - if (!irqchip_in_kernel(kvm) || ps->flags & KVM_PIT_FLAGS_HPET_LEGACY) + if (!irqchip_in_kernel(kvm) || + !ioapic_in_kernel(kvm) || + ps->flags & KVM_PIT_FLAGS_HPET_LEGACY) return; interval = muldiv64(val, NSEC_PER_SEC, KVM_PIT_FREQ); diff --git a/arch/x86/kvm/ioapic.h b/arch/x86/kvm/ioapic.h index ca0b0b4..d8cc54b 100644 --- a/arch/x86/kvm/ioapic.h +++ b/arch/x86/kvm/ioapic.h @@ -98,6 +98,15 @@ static inline struct kvm_ioapic *ioapic_irqchip(struct kvm *kvm) return kvm->arch.vioapic; } +static inline int ioapic_in_kernel(struct kvm *kvm) +{ + int ret; + + ret = (ioapic_irqchip(kvm) != NULL); + smp_rmb(); + return ret; +} + static inline bool kvm_ioapic_handles_vector(struct kvm *kvm, int vector) { struct kvm_ioapic *ioapic = kvm->arch.vioapic; diff --git a/arch/x86/kvm/irq.h b/arch/x86/kvm/irq.h index ad68c73..2f13dd5 100644 --- a/arch/x86/kvm/irq.h +++ b/arch/x86/kvm/irq.h @@ -83,11 +83,17 @@ static inline struct kvm_pic *pic_irqchip(struct kvm *kvm) return kvm->arch.vpic; } +static inline int irqchip_split(struct kvm *kvm) +{ + return kvm->arch.irqchip_split; +} + static inline int irqchip_in_kernel(struct kvm *kvm) { int ret; ret = (pic_irqchip(kvm) != NULL); + ret |= irqchip_split(kvm); smp_rmb(); return ret; } diff --git a/arch/x86/kvm/irq_comm.c b/arch/x86/kvm/irq_comm.c index 9efff9e..67f6b62 100644 --- a/arch/x86/kvm/irq_comm.c +++ b/arch/x86/kvm/irq_comm.c @@ -208,7 +208,7 @@ void kvm_free_irq_source_id(struct kvm *kvm, int irq_source_id) goto unlock; } clear_bit(irq_source_id, &kvm->arch.irq_sources_bitmap); - if (!irqchip_in_kernel(kvm)) + if (!ioapic_in_kernel(kvm)) goto unlock; kvm_ioapic_clear_all(kvm->arch.vioapic, irq_source_id); @@ -328,3 +328,10 @@ int kvm_setup_default_irq_routing(struct kvm *kvm) return kvm_set_irq_routing(kvm, default_routing, ARRAY_SIZE(default_routing), 0); } + +static const struct kvm_irq_routing_entry empty_routing[] = {}; + +int kvm_setup_empty_irq_routing(struct kvm *kvm) +{ + return kvm_set_irq_routing(kvm, empty_routing, 0, 0); +} diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 2a5ca97..536b79e 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -209,7 +209,8 @@ out: if (old) kfree_rcu(old, rcu); - kvm_vcpu_request_scan_ioapic(kvm); + if (!irqchip_split(kvm)) + kvm_vcpu_request_scan_ioapic(kvm); } static inline void apic_set_spiv(struct kvm_lapic *apic, u32 val) @@ -1838,7 +1839,8 @@ void kvm_apic_post_state_restore(struct kvm_vcpu *vcpu, kvm_x86_ops->hwapic_isr_update(vcpu->kvm, apic_find_highest_isr(apic)); kvm_make_request(KVM_REQ_EVENT, vcpu); - kvm_rtc_eoi_tracking_restore_one(vcpu); + if (!irqchip_split(vcpu->kvm)) + kvm_rtc_eoi_tracking_restore_one(vcpu); } void __kvm_migrate_apic_timer(struct kvm_vcpu *vcpu) @@ -1921,7 +1923,8 @@ static void apic_sync_pv_eoi_to_guest(struct kvm_vcpu *vcpu, /* Cache not set: could be safe but we don't bother. */ apic->highest_isr_cache == -1 || /* Need EOI to update ioapic. */ - kvm_ioapic_handles_vector(vcpu->kvm, apic->highest_isr_cache)) { + kvm_ioapic_handles_vector(vcpu->kvm, apic->highest_isr_cache) || + irqchip_split(vcpu->kvm)) { /* * PV EOI was disabled by apic_sync_pv_eoi_from_guest * so we need not do anything here. diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 4014a82..08203a1 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -948,7 +948,7 @@ static inline bool cpu_has_vmx_tpr_shadow(void) static inline bool vm_need_tpr_shadow(struct kvm *kvm) { - return (cpu_has_vmx_tpr_shadow()) && (irqchip_in_kernel(kvm)); + return (cpu_has_vmx_tpr_shadow()) && irqchip_in_kernel(kvm); } static inline bool cpu_has_secondary_exec_ctrls(void) @@ -9485,7 +9485,7 @@ static void prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12) /* vmcs12's VM_ENTRY_LOAD_IA32_EFER and VM_ENTRY_IA32E_MODE are * emulated by vmx_set_efer(), below. */ - vm_entry_controls_init(vmx, + vm_entry_controls_init(vmx, (vmcs12->vm_entry_controls & ~VM_ENTRY_LOAD_IA32_EFER & ~VM_ENTRY_IA32E_MODE) | (vmcs_config.vmentry_ctrl & ~VM_ENTRY_IA32E_MODE)); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 28076c2..6d4b4dc 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2461,6 +2461,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_TSC_DEADLINE_TIMER: case KVM_CAP_ENABLE_CAP_VM: case KVM_CAP_DISABLE_QUIRKS: + case KVM_CAP_SPLIT_IRQCHIP: #ifdef CONFIG_KVM_DEVICE_ASSIGNMENT case KVM_CAP_ASSIGN_DEV_IRQ: case KVM_CAP_PCI_2_3: @@ -3568,6 +3569,23 @@ static int kvm_vm_ioctl_enable_cap(struct kvm *kvm, kvm->arch.disabled_quirks = cap->args[0]; r = 0; break; + case KVM_CAP_SPLIT_IRQCHIP: { + mutex_lock(&kvm->lock); + r = -EEXIST; + if (irqchip_in_kernel(kvm)) + goto split_irqchip_unlock; + r = -EINVAL; + if (atomic_read(&kvm->online_vcpus)) + goto split_irqchip_unlock; + r = kvm_setup_empty_irq_routing(kvm); + if (r) + goto split_irqchip_unlock; + kvm->arch.irqchip_split = true; + r = 0; +split_irqchip_unlock: + mutex_unlock(&kvm->lock); + break; + } default: r = -EINVAL; break; @@ -3686,7 +3704,7 @@ long kvm_arch_vm_ioctl(struct file *filp, } r = -ENXIO; - if (!irqchip_in_kernel(kvm)) + if (!irqchip_in_kernel(kvm) || !ioapic_in_kernel(kvm)) goto get_irqchip_out; r = kvm_vm_ioctl_get_irqchip(kvm, chip); if (r) @@ -3710,7 +3728,7 @@ long kvm_arch_vm_ioctl(struct file *filp, } r = -ENXIO; - if (!irqchip_in_kernel(kvm)) + if (!irqchip_in_kernel(kvm) || !ioapic_in_kernel(kvm)) goto set_irqchip_out; r = kvm_vm_ioctl_set_irqchip(kvm, chip); if (r) @@ -3836,6 +3854,7 @@ long kvm_arch_vm_ioctl(struct file *filp, r = kvm_vm_ioctl_enable_cap(kvm, &cap); break; } + default: r = kvm_vm_ioctl_assigned_device(kvm, ioctl, arg); } diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index 133ea00..ffe1f4e 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -329,6 +329,7 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu); int kvm_vgic_vcpu_active_irq(struct kvm_vcpu *vcpu); #define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel)) +#define lapic_in_kernel(k) (irqchip_in_kernel(k)) #define vgic_initialized(k) (!!((k)->arch.vgic.nr_cpus)) #define vgic_ready(k) ((k)->arch.vgic.ready) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 51103f0..f7eab09 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1000,6 +1000,7 @@ static inline int mmu_notifier_retry(struct kvm *kvm, unsigned long mmu_seq) #endif int kvm_setup_default_irq_routing(struct kvm *kvm); +int kvm_setup_empty_irq_routing(struct kvm *kvm); int kvm_set_irq_routing(struct kvm *kvm, const struct kvm_irq_routing_entry *entries, unsigned nr, diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 9ef19eb..e4304d0 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -818,6 +818,7 @@ struct kvm_ppc_smmu_info { #define KVM_CAP_DISABLE_QUIRKS 116 #define KVM_CAP_X86_SMM 117 #define KVM_CAP_MULTI_ADDRESS_SPACE 118 +#define KVM_CAP_SPLIT_IRQCHIP 119 #ifdef KVM_CAP_IRQ_ROUTING