From patchwork Tue Mar 21 12:26:44 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 9636537 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A13AD602D6 for ; Tue, 21 Mar 2017 12:27:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 922582094F for ; Tue, 21 Mar 2017 12:27:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 85E7227E22; Tue, 21 Mar 2017 12:27:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B4BA42094F for ; Tue, 21 Mar 2017 12:27:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ydy1/9UPHAg5dqldJ0VvLIZYf3zAv6p0g0drJaomoAs=; b=iHGt0IUraSY6A+ NiOaJzanzG7NGSKM6zypikkZwbaVYYLw6G9GCNxA9+HOhaFcg0A8ROvNJzh0AerZ97WgUGDa3+1Xf +/0+Uih9GGUtZhGqrf+QgxNSPHRGfwQ54hFlYwQeI8c1RepJQ0qhqbjEvpx+QnL1HUil69YLEdYJV KuHmP2GCX2VqDVrKzD4LhHWf3GsG6JcsUi95NQNzaPRx6Zl883Z1P1BR0IkRHcMyWiuNNMsRv/0T1 /CgH4qwotv1LjvoxkWr/GSDdnspMoSou57AwTZMHs3sYrO5cXHNbfibgUd+PWSA9GCf8UXqdAr2gu cHNzJnBvxBjhn5MouIMw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1cqIsb-0004Sl-98; Tue, 21 Mar 2017 12:27:13 +0000 Received: from mail-wr0-x234.google.com ([2a00:1450:400c:c0c::234]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1cqIsV-0004RY-Em for linux-arm-kernel@lists.infradead.org; Tue, 21 Mar 2017 12:27:10 +0000 Received: by mail-wr0-x234.google.com with SMTP id l37so111046026wrc.1 for ; Tue, 21 Mar 2017 05:26:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=fWPQbP2ppgoeqqI9CkMFvWcJ4dBoBqrTt8EMgB7gr+A=; b=CJZUn5opF0MBbntQMphbirbUilqJf5AGNusiLi9onkzmn2Zt+IH0cCAxVu6RiDhF+j cU2ef0QFfr88ubgYbHnGC5VlFPlKErA4xXRct8p1+sS6rtucsgHXF8Vn28nIBkYMzUvE j3RJshE9a/rl2io6GxO0zkRmSzG35DjWuOjPs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=fWPQbP2ppgoeqqI9CkMFvWcJ4dBoBqrTt8EMgB7gr+A=; b=XMC3ZxrBYKiVxzEaKkxdXqwXmT4uLXE2Ez2hhAg7Q2pT5D79HaeQU+cvcitXHohINl t6nLDnWjVrYXBxfro9cuPeYIlABe9WMhtRQ8k+OsUHxVwJWKM9v+kAKCq2rLyqRSNcKo bN5M1Rey6rxTbGu6ky7wMoovAWB/jY+9k9TCSD+9BZqk/3plSnup+5omhwb9PT8rTOjU DdsqTOVlpt7AW/ZskCyVKZQCZsjUIkUfrDJBbefs78Nbw/e++o441qzq2vT/rMLP+EeS /LGXyyY2csYU6YnBEGHwev6qDC8wbib3MdER57UidIEMv/oFyhiGsXLiToq+qKLuxEAk tReQ== X-Gm-Message-State: AFeK/H0xRsD8MUQce37Yb4PfT6jcbJCKcwNXKVVPyaW4vzjlBXSfHVqu2fpZBIiKKrrfOncS X-Received: by 10.223.170.210 with SMTP id i18mr32925537wrc.114.1490099204913; Tue, 21 Mar 2017 05:26:44 -0700 (PDT) Received: from localhost (xd93ddc2d.cust.hiper.dk. [217.61.220.45]) by smtp.gmail.com with ESMTPSA id c35sm24757497wra.1.2017.03.21.05.26.43 (version=TLS1_2 cipher=AES128-SHA bits=128/128); Tue, 21 Mar 2017 05:26:44 -0700 (PDT) Date: Tue, 21 Mar 2017 13:26:44 +0100 From: Christoffer Dall To: Marc Zyngier Subject: Re: [PATCH 4/9] KVM: arm/arm64: vgic: Get rid of unnecessary process_maintenance operationjjjj Message-ID: <20170321122644.GD15920@cbox> References: <20170320105818.20481-1-cdall@linaro.org> <20170320105818.20481-5-cdall@linaro.org> <0da4b9c8-8631-e0f0-0ead-e9f65746b529@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <0da4b9c8-8631-e0f0-0ead-e9f65746b529@arm.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170321_052707_945564_1CF9AF1B X-CRM114-Status: GOOD ( 29.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Andre Przywara , Eric Auger , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, Christoffer Dall Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP On Tue, Mar 21, 2017 at 11:36:19AM +0000, Marc Zyngier wrote: > On 20/03/17 10:58, Christoffer Dall wrote: > > From: Christoffer Dall > > > > Since we always read back the LRs that we wrote to the guest and the > > MISR and EISR registers simply provide a summary of the configuration of > > the bits in the LRs, there is really no need to read back those status > > registers and process them. We can might as well just signal the > > notifyfd when folding the LR state and save some cycles in the process. > > > > Signed-off-by: Christoffer Dall > > --- > > virt/kvm/arm/vgic/vgic-v2.c | 55 ++++++++++----------------------------------- > > virt/kvm/arm/vgic/vgic-v3.c | 47 +++++++++++--------------------------- > > virt/kvm/arm/vgic/vgic.c | 8 +++---- > > virt/kvm/arm/vgic/vgic.h | 4 ++-- > > 4 files changed, 31 insertions(+), 83 deletions(-) > > > > diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c > > index dfe6e5e..0172754 100644 > > --- a/virt/kvm/arm/vgic/vgic-v2.c > > +++ b/virt/kvm/arm/vgic/vgic-v2.c > > @@ -22,20 +22,6 @@ > > > > #include "vgic.h" > > > > -/* > > - * Call this function to convert a u64 value to an unsigned long * bitmask > > - * in a way that works on both 32-bit and 64-bit LE and BE platforms. > > - * > > - * Warning: Calling this function may modify *val. > > - */ > > -static unsigned long *u64_to_bitmask(u64 *val) > > -{ > > -#if defined(CONFIG_CPU_BIG_ENDIAN) && BITS_PER_LONG == 32 > > - *val = (*val >> 32) | (*val << 32); > > -#endif > > - return (unsigned long *)val; > > -} > > - > > Getting rid of u64_to_bitmask makes me feel slightly better... > > > static inline void vgic_v2_write_lr(int lr, u32 val) > > { > > void __iomem *base = kvm_vgic_global_state.vctrl_base; > > @@ -51,38 +37,10 @@ void vgic_v2_init_lrs(void) > > vgic_v2_write_lr(i, 0); > > } > > > > -void vgic_v2_process_maintenance(struct kvm_vcpu *vcpu) > > +void vgic_v2_clear_uie(struct kvm_vcpu *vcpu) > > { > > struct vgic_v2_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v2; > > - > > - if (cpuif->vgic_misr & GICH_MISR_EOI) { > > - u64 eisr = cpuif->vgic_eisr; > > - unsigned long *eisr_bmap = u64_to_bitmask(&eisr); > > - int lr; > > - > > - for_each_set_bit(lr, eisr_bmap, kvm_vgic_global_state.nr_lr) { > > - u32 intid = cpuif->vgic_lr[lr] & GICH_LR_VIRTUALID; > > - > > - WARN_ON(cpuif->vgic_lr[lr] & GICH_LR_STATE); > > - > > - /* Only SPIs require notification */ > > - if (vgic_valid_spi(vcpu->kvm, intid)) > > - kvm_notify_acked_irq(vcpu->kvm, 0, > > - intid - VGIC_NR_PRIVATE_IRQS); > > - } > > - } > > - > > - /* check and disable underflow maintenance IRQ */ > > cpuif->vgic_hcr &= ~GICH_HCR_UIE; > > - > > - /* > > - * In the next iterations of the vcpu loop, if we sync the > > - * vgic state after flushing it, but before entering the guest > > - * (this happens for pending signals and vmid rollovers), then > > - * make sure we don't pick up any old maintenance interrupts > > - * here. > > - */ > > - cpuif->vgic_eisr = 0; > > } > > > > void vgic_v2_set_underflow(struct kvm_vcpu *vcpu) > > @@ -92,6 +50,12 @@ void vgic_v2_set_underflow(struct kvm_vcpu *vcpu) > > cpuif->vgic_hcr |= GICH_HCR_UIE; > > } > > > > +static bool lr_signals_eoi_mi(u32 lr_val) > > +{ > > + return !(lr_val & GICH_LR_STATE) && (lr_val & GICH_LR_EOI) && > > + !(lr_val & GICH_LR_HW); > > +} > > + > > /* > > * transfer the content of the LRs back into the corresponding ap_list: > > * - active bit is transferred as is > > @@ -109,6 +73,11 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu) > > u32 intid = val & GICH_LR_VIRTUALID; > > struct vgic_irq *irq; > > > > + /* Notify fds when the guest EOI'ed a level-triggered SPI */ > > + if (lr_signals_eoi_mi(val) && vgic_valid_spi(vcpu->kvm, intid)) > > + kvm_notify_acked_irq(vcpu->kvm, 0, > > + intid - VGIC_NR_PRIVATE_IRQS); > > + > > irq = vgic_get_irq(vcpu->kvm, vcpu, intid); > > > > spin_lock(&irq->irq_lock); > > diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c > > index 3d7796c..7880c5c 100644 > > --- a/virt/kvm/arm/vgic/vgic-v3.c > > +++ b/virt/kvm/arm/vgic/vgic-v3.c > > @@ -21,42 +21,9 @@ > > > > #include "vgic.h" > > > > -void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu) > > +void vgic_v3_clear_uie(struct kvm_vcpu *vcpu) > > { > > struct vgic_v3_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v3; > > - u32 model = vcpu->kvm->arch.vgic.vgic_model; > > - > > - if (cpuif->vgic_misr & ICH_MISR_EOI) { > > - unsigned long eisr_bmap = cpuif->vgic_eisr; > > - int lr; > > - > > - for_each_set_bit(lr, &eisr_bmap, kvm_vgic_global_state.nr_lr) { > > - u32 intid; > > - u64 val = cpuif->vgic_lr[lr]; > > - > > - if (model == KVM_DEV_TYPE_ARM_VGIC_V3) > > - intid = val & ICH_LR_VIRTUAL_ID_MASK; > > - else > > - intid = val & GICH_LR_VIRTUALID; > > - > > - WARN_ON(cpuif->vgic_lr[lr] & ICH_LR_STATE); > > - > > - /* Only SPIs require notification */ > > - if (vgic_valid_spi(vcpu->kvm, intid)) > > - kvm_notify_acked_irq(vcpu->kvm, 0, > > - intid - VGIC_NR_PRIVATE_IRQS); > > - } > > - > > - /* > > - * In the next iterations of the vcpu loop, if we sync > > - * the vgic state after flushing it, but before > > - * entering the guest (this happens for pending > > - * signals and vmid rollovers), then make sure we > > - * don't pick up any old maintenance interrupts here. > > - */ > > - cpuif->vgic_eisr = 0; > > - } > > - > > cpuif->vgic_hcr &= ~ICH_HCR_UIE; > > } > > > > @@ -67,6 +34,12 @@ void vgic_v3_set_underflow(struct kvm_vcpu *vcpu) > > cpuif->vgic_hcr |= ICH_HCR_UIE; > > } > > > > +static bool lr_signals_eoi_mi(u64 lr_val) > > +{ > > + return !(lr_val & ICH_LR_STATE) && (lr_val & ICH_LR_EOI) && > > + !(lr_val & ICH_LR_HW); > > +} > > + > > void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu) > > { > > struct vgic_v3_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v3; > > @@ -82,6 +55,12 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu) > > intid = val & ICH_LR_VIRTUAL_ID_MASK; > > else > > intid = val & GICH_LR_VIRTUALID; > > + > > + /* Notify fds when the guest EOI'ed a level-triggered IRQ */ > > + if (lr_signals_eoi_mi(val) && vgic_valid_spi(vcpu->kvm, intid)) > > + kvm_notify_acked_irq(vcpu->kvm, 0, > > + intid - VGIC_NR_PRIVATE_IRQS); > > + > > irq = vgic_get_irq(vcpu->kvm, vcpu, intid); > > if (!irq) /* An LPI could have been unmapped. */ > > continue; > > diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c > > index 1436c2e..f5b3c25 100644 > > --- a/virt/kvm/arm/vgic/vgic.c > > +++ b/virt/kvm/arm/vgic/vgic.c > > @@ -527,12 +527,12 @@ static void vgic_prune_ap_list(struct kvm_vcpu *vcpu) > > spin_unlock(&vgic_cpu->ap_list_lock); > > } > > > > -static inline void vgic_process_maintenance_interrupt(struct kvm_vcpu *vcpu) > > +static inline void vgic_clear_uie(struct kvm_vcpu *vcpu) > > { > > if (kvm_vgic_global_state.type == VGIC_V2) > > - vgic_v2_process_maintenance(vcpu); > > + vgic_v2_clear_uie(vcpu); > > else > > - vgic_v3_process_maintenance(vcpu); > > + vgic_v3_clear_uie(vcpu); > > } > > > > static inline void vgic_fold_lr_state(struct kvm_vcpu *vcpu) > > @@ -642,7 +642,7 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu) > > if (unlikely(!vgic_initialized(vcpu->kvm))) > > return; > > > > - vgic_process_maintenance_interrupt(vcpu); > > + vgic_clear_uie(vcpu); > > vgic_fold_lr_state(vcpu); > > nit: If you moved vgic_*_clear_uie() into vgic_*_fold_lr_state, you > could get rid of this call, extra prototypes, and save yourself the > extra check. Not a big deal though. > I thought about this, but the problem is that strictly speaking, you don't need to consume any LRs to set the underflow, you just need enough in the AP list, and it's theoretically possible that you have nr_lrs+1 interrupts in the AP list which have all been migrated away. So, it would require something like this including a rework in flush_lr_state to work with the following patch which only calls vgic_fold_lr_state when there are in fact anything live in LRs: What do you think? > > vgic_prune_ap_list(vcpu); > > > > diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h > > index 3f64220..72ba9ee 100644 > > --- a/virt/kvm/arm/vgic/vgic.h > > +++ b/virt/kvm/arm/vgic/vgic.h > > @@ -112,7 +112,7 @@ void vgic_kick_vcpus(struct kvm *kvm); > > int vgic_check_ioaddr(struct kvm *kvm, phys_addr_t *ioaddr, > > phys_addr_t addr, phys_addr_t alignment); > > > > -void vgic_v2_process_maintenance(struct kvm_vcpu *vcpu); > > +void vgic_v2_clear_uie(struct kvm_vcpu *vcpu); > > void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu); > > void vgic_v2_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr); > > void vgic_v2_clear_lr(struct kvm_vcpu *vcpu, int lr); > > @@ -143,7 +143,7 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq) > > kref_get(&irq->refcount); > > } > > > > -void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu); > > +void vgic_v3_clear_uie(struct kvm_vcpu *vcpu); > > void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu); > > void vgic_v3_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr); > > void vgic_v3_clear_lr(struct kvm_vcpu *vcpu, int lr); > > > > Reviewed-by: Marc Zyngier > Thanks, -Christoffer diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c index e5cf930..4b1feef 100644 --- a/virt/kvm/arm/vgic/vgic.c +++ b/virt/kvm/arm/vgic/vgic.c @@ -601,10 +601,8 @@ static void vgic_flush_lr_state(struct kvm_vcpu *vcpu) DEBUG_SPINLOCK_BUG_ON(!spin_is_locked(&vgic_cpu->ap_list_lock)); - if (compute_ap_list_depth(vcpu) > kvm_vgic_global_state.nr_lr) { - vgic_set_underflow(vcpu); + if (compute_ap_list_depth(vcpu) > kvm_vgic_global_state.nr_lr) vgic_sort_ap_list(vcpu); - } list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) { spin_lock(&irq->irq_lock); @@ -623,8 +621,12 @@ static void vgic_flush_lr_state(struct kvm_vcpu *vcpu) next: spin_unlock(&irq->irq_lock); - if (count == kvm_vgic_global_state.nr_lr) + if (count == kvm_vgic_global_state.nr_lr) { + if (!list_is_last(&irq->ap_list, + &vgic_cpu->ap_list_head)) + vgic_set_underflow(vcpu); break; + } } vcpu->arch.vgic_cpu.used_lrs = count;