From patchwork Fri Mar 3 02:35:43 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chao Gao X-Patchwork-Id: 9602415 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2BAB16016C for ; Fri, 3 Mar 2017 09:40:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1D467285AF for ; Fri, 3 Mar 2017 09:40:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 11E65285D6; Fri, 3 Mar 2017 09:40:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=BAYES_00, DATE_IN_PAST_06_12, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8F127285AF for ; Fri, 3 Mar 2017 09:40:22 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cjjex-0001Zm-8t; Fri, 03 Mar 2017 09:37:59 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cjjew-0001Zf-0x for xen-devel@lists.xen.org; Fri, 03 Mar 2017 09:37:58 +0000 Received: from [85.158.143.35] by server-10.bemta-6.messagelabs.com id A2/4F-13192-57939B85; Fri, 03 Mar 2017 09:37:57 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrKLMWRWlGSWpSXmKPExsXS1tbhqFtouTP C4PkHaYslHxezODB6HN39mymAMYo1My8pvyKBNWPisUWMBb/UKg5flGpg3CrbxcjFISQwnVGi 5eB2ti5GTg4JAV6JI8tmsHYxcgDZARIXjqeBhIUEyiVub9rMAmKzCShLXPzaC1YuIiAtce3zZ UaQOcwCU5gkPq/7xQySEBaIlOj6tYYVxGYRUJXY2niWCcTmFXCS+HbkJivELgWJKQ/fM09g5F 7AyLCKUaM4tagstUjX0FAvqSgzPaMkNzEzR9fQwEwvN7W4ODE9NScxqVgvOT93EyPQuwxAsIP x07KAQ4ySHExKorzHTXdECPEl5adUZiQWZ8QXleakFh9ilOHgUJLgtQgFygkWpaanVqRl5gDD DCYtwcGjJMLrCZLmLS5IzC3OTIdInWJUlBLn/RoClBAASWSU5sG1wUL7EqOslDAvI9AhQjwFq UW5mSWo8q8YxTkYlYR5o0HG82TmlcBNfwW0mAlo8QuVrSCLSxIRUsB4YTt/1XXT2ZzM/KuMeg dPFJ87wqI1++UahcSwGW/O7X03Wy31V//jVH1O2QWP3dOS11QtP6lvuuLD261Zk+0Oi6pU1D7 nmLuUrZzjjoxax/sfEvolL4Svv6v8e33e3hnnylPil6u7tly4v31By/st/ucn3V/u/GKDU93r tScql4jOjXLb85l92X4lluKMREMt5qLiRACSUYcPaAIAAA== X-Env-Sender: chao.gao@intel.com X-Msg-Ref: server-8.tower-21.messagelabs.com!1488533871!60345845!1 X-Originating-IP: [134.134.136.65] X-SpamReason: No, hits=1.3 required=7.0 tests=BODY_RANDOM_LONG, DATE_IN_PAST_06_12 X-StarScan-Received: X-StarScan-Version: 9.4.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 18568 invoked from network); 3 Mar 2017 09:37:53 -0000 Received: from mga03.intel.com (HELO mga03.intel.com) (134.134.136.65) by server-8.tower-21.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 3 Mar 2017 09:37:53 -0000 Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Mar 2017 01:37:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.35,236,1484035200"; d="scan'208"; a="1117945661" Received: from skl-2s3.sh.intel.com ([10.239.48.35]) by fmsmga001.fm.intel.com with ESMTP; 03 Mar 2017 01:37:48 -0800 From: Chao Gao To: xen-devel@lists.xen.org Date: Fri, 3 Mar 2017 10:35:43 +0800 Message-Id: <1488508543-30295-1-git-send-email-chao.gao@intel.com> X-Mailer: git-send-email 1.8.3.1 Cc: yang.zhang.wz@gmail.com, "Xuquan \(Quan Xu\)" , quan.xu0@gmail.com, Jun Nakajima , Andrew Cooper , Kevin Tian , Jan Beulich , Chao Gao Subject: [Xen-devel] [PATCH v4] x86/apicv: Fix wrong IPI suppression during posted interrupt delivery X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP __vmx_deliver_posted_interrupt() wrongly used a softirq bit to decide whether to suppress an IPI. Its logic was: the first time an IPI was sent, we set the softirq bit. Next time, we would check that softirq bit before sending another IPI. If the 1st IPI arrived at the pCPU which was in non-root mode, the hardware would consume the IPI and sync PIR to vIRR. During the process, no one (both hardware and software) will clear the softirq bit. As a result, the following IPI would be wrongly suppressed. This patch discards the suppression check, always sending an IPI. The softirq also need to be raised. But there is a little change. This patch moves the place where we raise a softirq for 'cpu != smp_processor_id()' case to the IPI interrupt handler. Namely, don't raise a softirq for this case and set the interrupt handler to pi_notification_interrupt()(in which a softirq is raised) regardless of VT-d PI enabled or not. The only difference is when an IPI arrives at the pCPU which is happened in non-root mode, the code will not raise a useless softirq since the IPI is consumed by hardware rather than raise a softirq unconditionally. Signed-off-by: Quan Xu Signed-off-by: Chao Gao Acked-by: Kevin Tian --- v4: - Replace patch v3 title "Enhance posted-interrupt processing" with the current title. - Desciption changes - Fix a compiler error --- xen/arch/x86/hvm/vmx/vmx.c | 50 +++++++++++++++++++++++++++++++++++++++------- 1 file changed, 43 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index 5b1717d..1a0e130 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -1842,13 +1842,53 @@ static void __vmx_deliver_posted_interrupt(struct vcpu *v) bool_t running = v->is_running; vcpu_unblock(v); + /* + * Just like vcpu_kick(), nothing is needed for the following two cases: + * 1. The target vCPU is not running, meaning it is blocked or runnable. + * 2. The target vCPU is the current vCPU and we're in non-interrupt + * context. + */ if ( running && (in_irq() || (v != current)) ) { + /* + * Note: Only two cases will reach here: + * 1. The target vCPU is running on other pCPU. + * 2. The target vCPU is the current vCPU. + * + * Note2: Don't worry the v->processor may change. The vCPU being + * moved to another processor is guaranteed to sync PIR to vIRR, + * due to the involved scheduling cycle. + */ unsigned int cpu = v->processor; - if ( !test_and_set_bit(VCPU_KICK_SOFTIRQ, &softirq_pending(cpu)) - && (cpu != smp_processor_id()) ) + /* + * For case 1, we send an IPI to the pCPU. When an IPI arrives, the + * target vCPU maybe is running in non-root mode, running in root + * mode, runnable or blocked. If the target vCPU is running in + * non-root mode, the hardware will sync PIR to vIRR for + * 'posted_intr_vector' is special to the pCPU. If the target vCPU is + * running in root-mode, the interrupt handler starts to run. + * Considering an IPI may arrive in the window between the call to + * vmx_intr_assist() and interrupts getting disabled, the interrupt + * handler should raise a softirq to ensure events will be delivered + * in time. If the target vCPU is runnable, it will sync PIR to + * vIRR next time it is chose to run. In this case, a IPI and a + * softirq is sent to a wrong vCPU which will not have any adverse + * effect. If the target vCPU is blocked, since vcpu_block() checks + * whether there is an event to be delivered through + * local_events_need_delivery() just after blocking, the vCPU must + * have synced PIR to vIRR. Similarly, there is a IPI and a softirq + * sent to a wrong vCPU. + */ + if ( cpu != smp_processor_id() ) send_IPI_mask(cpumask_of(cpu), posted_intr_vector); + /* + * For case 2, raising a softirq ensures PIR will be synced to vIRR. + * As any softirq will do, as an optimization we only raise one if + * none is pending already. + */ + else if ( !softirq_pending(cpu) ) + raise_softirq(VCPU_KICK_SOFTIRQ); } } @@ -2281,13 +2321,9 @@ const struct hvm_function_table * __init start_vmx(void) if ( cpu_has_vmx_posted_intr_processing ) { + alloc_direct_apic_vector(&posted_intr_vector, pi_notification_interrupt); if ( iommu_intpost ) - { - alloc_direct_apic_vector(&posted_intr_vector, pi_notification_interrupt); alloc_direct_apic_vector(&pi_wakeup_vector, pi_wakeup_interrupt); - } - else - alloc_direct_apic_vector(&posted_intr_vector, event_check_interrupt); } else {