From patchwork Thu Jul 16 14:03:39 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gleb Natapov X-Patchwork-Id: 35877 Received: from vger.kernel.org (vger.kernel.org [209.132.176.167]) by demeter.kernel.org (8.14.2/8.14.2) with ESMTP id n6GE46Ht031991 for ; Thu, 16 Jul 2009 14:04:07 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932357AbZGPODz (ORCPT ); Thu, 16 Jul 2009 10:03:55 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S932286AbZGPODz (ORCPT ); Thu, 16 Jul 2009 10:03:55 -0400 Received: from mx2.redhat.com ([66.187.237.31]:40153 "EHLO mx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932091AbZGPODv (ORCPT ); Thu, 16 Jul 2009 10:03:51 -0400 Received: from int-mx2.corp.redhat.com (int-mx2.corp.redhat.com [172.16.27.26]) by mx2.redhat.com (8.13.8/8.13.8) with ESMTP id n6GE3pr3025898 for ; Thu, 16 Jul 2009 10:03:51 -0400 Received: from ns3.rdu.redhat.com (ns3.rdu.redhat.com [10.11.255.199]) by int-mx2.corp.redhat.com (8.13.1/8.13.1) with ESMTP id n6GE3j73014568; Thu, 16 Jul 2009 10:03:45 -0400 Received: from dhcp-1-237.tlv.redhat.com (dhcp-1-237.tlv.redhat.com [10.35.1.237]) by ns3.rdu.redhat.com (8.13.8/8.13.8) with ESMTP id n6GE3h0v001308; Thu, 16 Jul 2009 10:03:44 -0400 Received: by dhcp-1-237.tlv.redhat.com (Postfix, from userid 13519) id 4238D18D489; Thu, 16 Jul 2009 17:03:40 +0300 (IDT) From: Gleb Natapov To: kvm@vger.kernel.org Cc: mtosatti@redhat.com Subject: [PATCH 11/11] Introduce MSI message sending interface that bypass IRQ routing. Date: Thu, 16 Jul 2009 17:03:39 +0300 Message-Id: <1247753019-11412-12-git-send-email-gleb@redhat.com> In-Reply-To: <1247753019-11412-1-git-send-email-gleb@redhat.com> References: <1247753019-11412-1-git-send-email-gleb@redhat.com> X-Scanned-By: MIMEDefang 2.58 on 172.16.27.26 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Sending of MSI using IRQ routing is an artificial concept and potentially big number of MSIs (2048 per device) make it also inefficient. This patch adds an interface to inject MSI messages from userspace to lapic logic directly. The patch also reduces the maximum number of IRQ routing entries to 128 since MSIs will no longer go there and 128 entries cover 5 ioapics and this ought to be enough for anybody. Signed-off-by: Gleb Natapov --- arch/ia64/kvm/kvm-ia64.c | 26 ++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 26 ++++++++++++++++++++++++++ include/linux/kvm.h | 10 ++++++++-- include/linux/kvm_host.h | 3 ++- virt/kvm/irq_comm.c | 23 ++++++++++++++--------- 5 files changed, 76 insertions(+), 12 deletions(-) diff --git a/arch/ia64/kvm/kvm-ia64.c b/arch/ia64/kvm/kvm-ia64.c index 8f1fc3a..c136085 100644 --- a/arch/ia64/kvm/kvm-ia64.c +++ b/arch/ia64/kvm/kvm-ia64.c @@ -195,6 +195,7 @@ int kvm_dev_ioctl_check_extension(long ext) case KVM_CAP_IRQCHIP: case KVM_CAP_MP_STATE: case KVM_CAP_IRQ_INJECT_STATUS: + case KVM_CAP_MSI_MSG: r = 1; break; case KVM_CAP_COALESCED_MMIO: @@ -1010,6 +1011,31 @@ long kvm_arch_vm_ioctl(struct file *filp, } break; } + case KVM_MSI_MSG: { + struct kvm_irq_routing_msi msg; + struct msi_msg msi; + + r = -EFAULT; + if (copy_from_user(&msg, argp, sizeof msg)) + goto out; + r = -EINVAL; + if (!irqchip_in_kernel(kvm)) + goto out; + if (msg.flags) + goto out; + + msi.address_lo = msg.address_lo; + msi.address_hi = msg.address_hi; + msi.data = msg.data; + + msg.status = kvm_set_msi(kvm, &msi); + if (copy_to_user(argp, &msg, sizeof msg)) { + r = -EFAULT; + goto out; + } + r = 0; + break; + } case KVM_GET_IRQCHIP: { /* 0: PIC master, 1: PIC slave, 2: IOAPIC */ struct kvm_irqchip chip; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 40adac5..a8815f8 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1208,6 +1208,7 @@ int kvm_dev_ioctl_check_extension(long ext) case KVM_CAP_IOEVENTFD: case KVM_CAP_PIT2: case KVM_CAP_PIT_STATE2: + case KVM_CAP_MSI_MSG: r = 1; break; case KVM_CAP_COALESCED_MMIO: @@ -2273,6 +2274,31 @@ long kvm_arch_vm_ioctl(struct file *filp, } break; } + case KVM_MSI_MSG: { + struct kvm_irq_routing_msi msg; + struct msi_msg msi; + + r = -EFAULT; + if (copy_from_user(&msg, argp, sizeof msg)) + goto out; + r = -EINVAL; + if (!irqchip_in_kernel(kvm)) + goto out; + if (msg.flags) + goto out; + + msi.address_lo = msg.address_lo; + msi.address_hi = msg.address_hi; + msi.data = msg.data; + + msg.status = kvm_set_msi(kvm, &msi); + if (copy_to_user(argp, &msg, sizeof msg)) { + r = -EFAULT; + goto out; + } + r = 0; + break; + } case KVM_GET_IRQCHIP: { /* 0: PIC master, 1: PIC slave, 2: IOAPIC */ struct kvm_irqchip *chip = kmalloc(sizeof(*chip), GFP_KERNEL); diff --git a/include/linux/kvm.h b/include/linux/kvm.h index 230a91a..19bc586 100644 --- a/include/linux/kvm.h +++ b/include/linux/kvm.h @@ -435,6 +435,7 @@ struct kvm_ioeventfd { #define KVM_CAP_PIT_STATE2 35 #endif #define KVM_CAP_IOEVENTFD 36 +#define KVM_CAP_MSI_MSG 37 #ifdef KVM_CAP_IRQ_ROUTING @@ -446,8 +447,11 @@ struct kvm_irq_routing_irqchip { struct kvm_irq_routing_msi { __u32 address_lo; __u32 address_hi; - __u32 data; - __u32 pad; + union { + __u32 data; + __s32 status; + }; + __u32 flags; }; /* gsi routing entry types */ @@ -591,6 +595,8 @@ struct kvm_irqfd { #define KVM_X86_SETUP_MCE _IOW(KVMIO, 0x9c, __u64) #define KVM_X86_GET_MCE_CAP_SUPPORTED _IOR(KVMIO, 0x9d, __u64) #define KVM_X86_SET_MCE _IOW(KVMIO, 0x9e, struct kvm_x86_mce) +#define KVM_MSI_MSG \ + _IOWR(KVMIO, 0x9f, struct kvm_irq_routing_msi) /* * Deprecated interfaces diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 2715e59..f711a7d 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -406,6 +406,7 @@ void kvm_get_intr_delivery_bitmask(struct kvm_ioapic *ioapic, #endif int kvm_set_irq(struct kvm *kvm, int irq_source_id, int irq, int level); int kvm_notifier_set_irq(struct kvm *kvm, int irq_source_id, int irq, int level); +int kvm_set_msi(struct kvm *kvm, struct msi_msg *msg); void kvm_notify_acked_irq(struct kvm *kvm, unsigned irqchip, unsigned pin); void kvm_register_irq_ack_notifier(struct kvm *kvm, struct kvm_irq_ack_notifier *kian); @@ -523,7 +524,7 @@ static inline int mmu_notifier_retry(struct kvm_vcpu *vcpu, unsigned long mmu_se #ifdef CONFIG_HAVE_KVM_IRQCHIP -#define KVM_MAX_IRQ_ROUTES 1024 +#define KVM_MAX_IRQ_ROUTES 128 int kvm_setup_default_irq_routing(struct kvm *kvm); int kvm_set_irq_routing(struct kvm *kvm, diff --git a/virt/kvm/irq_comm.c b/virt/kvm/irq_comm.c index ae11114..2ce1c93 100644 --- a/virt/kvm/irq_comm.c +++ b/virt/kvm/irq_comm.c @@ -49,6 +49,12 @@ static int kvm_set_ioapic_irq(struct kvm_kernel_irq_routing_entry *e, notifier); } +static int kvm_set_msi_irq(struct kvm_kernel_irq_routing_entry *e, + struct kvm *kvm, int level, bool notifier) +{ + return kvm_set_msi(kvm, &e->msi); +} + inline static bool kvm_is_dm_lowest_prio(struct kvm_lapic_irq *irq) { #ifdef CONFIG_IA64 @@ -95,20 +101,19 @@ int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src, return r; } -static int kvm_set_msi(struct kvm_kernel_irq_routing_entry *e, - struct kvm *kvm, int level) +int kvm_set_msi(struct kvm *kvm, struct msi_msg *msi) { struct kvm_lapic_irq irq; - trace_kvm_msi_set_irq(e->msi.address_lo, e->msi.data); + trace_kvm_msi_set_irq(msi->address_lo, msi->data); - irq.dest_id = (e->msi.address_lo & + irq.dest_id = (msi->address_lo & MSI_ADDR_DEST_ID_MASK) >> MSI_ADDR_DEST_ID_SHIFT; - irq.vector = (e->msi.data & + irq.vector = (msi->data & MSI_DATA_VECTOR_MASK) >> MSI_DATA_VECTOR_SHIFT; - irq.dest_mode = (1 << MSI_ADDR_DEST_MODE_SHIFT) & e->msi.address_lo; - irq.trig_mode = (1 << MSI_DATA_TRIGGER_SHIFT) & e->msi.data; - irq.delivery_mode = e->msi.data & 0x700; + irq.dest_mode = (1 << MSI_ADDR_DEST_MODE_SHIFT) & msi->address_lo; + irq.trig_mode = (1 << MSI_DATA_TRIGGER_SHIFT) & msi->data; + irq.delivery_mode = msi->data & 0x700; irq.level = 1; irq.shorthand = 0; @@ -319,7 +324,7 @@ static int setup_routing_entry(struct kvm_irq_routing_table *rt, rt->chip[ue->u.irqchip.irqchip][e->irqchip.pin] = ue->gsi; break; case KVM_IRQ_ROUTING_MSI: - e->set = kvm_set_msi; + e->set = kvm_set_msi_irq; e->msi.address_lo = ue->u.msi.address_lo; e->msi.address_hi = ue->u.msi.address_hi; e->msi.data = ue->u.msi.data;