From patchwork Mon Nov 23 06:54:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Shenming Lu X-Patchwork-Id: 11924439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39362C63777 for ; Mon, 23 Nov 2020 06:55:38 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B284C20773 for ; Mon, 23 Nov 2020 06:55:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="sPPO0gGn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B284C20773 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-ID:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=UIx5pCQcIUGNuNDsePTjQ71idcYtLK7Yi8L9EFzrcG0=; b=sPPO0gGnaNBvdsk2NUJunJc+z +J1dhkBM9FZuSHQJO2Ez2/+y0VvatKLMZt0nJaKBdY5X6U7yOW9hYXjZbakiBoueGOQH8mOZR0Px4 h86ZLmTFgpVl6vO8N966AzNC2anpN/SYPOnDDHuIhIGY3Tyh15aBfy41UqpKLiEuKWfuABzeQQ2kO H9CFEr2PqzHjAVG5Rfg5UB+kVbBizgDAeaWGoBPlqK+/NV9ZUYZ4KLaUCAXRP29uyVgevEDh44pz6 xdT82NtPj9kMXhL6N44+c7P3uyES4SFhjpWJ4NZ3ERYl0oRNy0LbM1oymk1SaVUuoowmFevVbopFs U8jhSF8/A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kh5kl-0007x5-LO; Mon, 23 Nov 2020 06:55:11 +0000 Received: from szxga05-in.huawei.com ([45.249.212.191]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kh5kf-0007vA-2L for linux-arm-kernel@lists.infradead.org; Mon, 23 Nov 2020 06:55:06 +0000 Received: from DGGEMS405-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4CfdDn1KpJzhg2v; Mon, 23 Nov 2020 14:54:33 +0800 (CST) Received: from DESKTOP-7FEPK9S.china.huawei.com (10.174.187.74) by DGGEMS405-HUB.china.huawei.com (10.3.19.205) with Microsoft SMTP Server id 14.3.487.0; Mon, 23 Nov 2020 14:54:43 +0800 From: Shenming Lu To: Marc Zyngier , James Morse , "Julien Thierry" , Suzuki K Poulose , Eric Auger , , , , , Christoffer Dall Subject: [RFC PATCH v1 2/4] KVM: arm64: GICv4.1: Try to save hw pending state in save_pending_tables Date: Mon, 23 Nov 2020 14:54:08 +0800 Message-ID: <20201123065410.1915-3-lushenming@huawei.com> X-Mailer: git-send-email 2.27.0.windows.1 In-Reply-To: <20201123065410.1915-1-lushenming@huawei.com> References: <20201123065410.1915-1-lushenming@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.187.74] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201123_015505_638571_F001E9E5 X-CRM114-Status: GOOD ( 16.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Neo Jia , Cornelia Huck , Kirti Wankhede , lushenming@huawei.com, Alex Williamson , yuzenghui@huawei.com, wanghaibin.wang@huawei.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org After pausing all vCPUs and devices capable of interrupting, in order to save the information of all interrupts, besides flushing the pending states in kvm’s vgic, we also try to flush the states of VLPIs in the virtual pending tables into guest RAM, but we need to have GICv4.1 and safely unmap the vPEs first. Signed-off-by: Shenming Lu --- arch/arm64/kvm/vgic/vgic-v3.c | 62 +++++++++++++++++++++++++++++++---- 1 file changed, 56 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c index 9cdf39a94a63..e1b3aa4b2b12 100644 --- a/arch/arm64/kvm/vgic/vgic-v3.c +++ b/arch/arm64/kvm/vgic/vgic-v3.c @@ -1,6 +1,8 @@ // SPDX-License-Identifier: GPL-2.0-only #include +#include +#include #include #include #include @@ -356,6 +358,39 @@ int vgic_v3_lpi_sync_pending_status(struct kvm *kvm, struct vgic_irq *irq) return 0; } +/* + * With GICv4.1, we can get the VLPI's pending state after unmapping + * the vPE. The deactivation of the doorbell interrupt will trigger + * the unmapping of the associated vPE. + */ +static void get_vlpi_state_pre(struct vgic_dist *dist) +{ + struct irq_desc *desc; + int i; + + if (!kvm_vgic_global_state.has_gicv4_1) + return; + + for (i = 0; i < dist->its_vm.nr_vpes; i++) { + desc = irq_to_desc(dist->its_vm.vpes[i]->irq); + irq_domain_deactivate_irq(irq_desc_get_irq_data(desc)); + } +} + +static void get_vlpi_state_post(struct vgic_dist *dist) +{ + struct irq_desc *desc; + int i; + + if (!kvm_vgic_global_state.has_gicv4_1) + return; + + for (i = 0; i < dist->its_vm.nr_vpes; i++) { + desc = irq_to_desc(dist->its_vm.vpes[i]->irq); + irq_domain_activate_irq(irq_desc_get_irq_data(desc), false); + } +} + /** * vgic_v3_save_pending_tables - Save the pending tables into guest RAM * kvm lock and all vcpu lock must be held @@ -365,14 +400,17 @@ int vgic_v3_save_pending_tables(struct kvm *kvm) struct vgic_dist *dist = &kvm->arch.vgic; struct vgic_irq *irq; gpa_t last_ptr = ~(gpa_t)0; - int ret; + int ret = 0; u8 val; + get_vlpi_state_pre(dist); + list_for_each_entry(irq, &dist->lpi_list_head, lpi_list) { int byte_offset, bit_nr; struct kvm_vcpu *vcpu; gpa_t pendbase, ptr; bool stored; + bool is_pending = irq->pending_latch; vcpu = irq->target_vcpu; if (!vcpu) @@ -387,24 +425,36 @@ int vgic_v3_save_pending_tables(struct kvm *kvm) if (ptr != last_ptr) { ret = kvm_read_guest_lock(kvm, ptr, &val, 1); if (ret) - return ret; + goto out; last_ptr = ptr; } stored = val & (1U << bit_nr); - if (stored == irq->pending_latch) + + /* also flush hw pending state */ + if (irq->hw) { + WARN_RATELIMIT(irq_get_irqchip_state(irq->host_irq, + IRQCHIP_STATE_PENDING, &is_pending), + "IRQ %d", irq->host_irq); + } + + if (stored == is_pending) continue; - if (irq->pending_latch) + if (is_pending) val |= 1 << bit_nr; else val &= ~(1 << bit_nr); ret = kvm_write_guest_lock(kvm, ptr, &val, 1); if (ret) - return ret; + goto out; } - return 0; + +out: + get_vlpi_state_post(dist); + + return ret; } /**