From patchwork Mon Nov 13 09:17:50 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 10055419 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 696886029B for ; Mon, 13 Nov 2017 09:35:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4127228FD5 for ; Mon, 13 Nov 2017 09:35:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 32A5C29191; Mon, 13 Nov 2017 09:35:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9DB0F28FD5 for ; Mon, 13 Nov 2017 09:35:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=owtPDhyPm4IJA9tpEAXlTDIsjnca8hr7Nv84vcpcQRQ=; b=Z7A5xOIiqyzWDf/pSoJ3KVBw/z oHC3haFO/pmvoy5s07iQn8VLI2OJi0ND8OJFXdBsDlC91YUWIEhoTap0a16GJaG9bBFtHpeC2Gu+t 7Qx4ldO2Rn3r2oNim3aYk6iUVFfAC4BZxlZBFWxA1qlgnzIc2Z16aohAzHZThxI4Hm9id7r54Pxp5 cEbPTNtpPSzN2oByDLpEIWIODCmbCvjmhtTRoookwwzUbXrnkzb2rDw99pJgmrx0S5QWYZ3DoFB46 lJnhhpjuiZWSfItgMyr/K/lyWC6xbwsIwM162GhcJTewOOcl+MOl4hhBxmqaDk4eJ6Tq8bQk+LAkE fUACNhmg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1eEB93-0003UG-3B; Mon, 13 Nov 2017 09:35:09 +0000 Received: from merlin.infradead.org ([2001:8b0:10b:1231::1]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1eEB8R-0002W7-LW for linux-arm-kernel@bombadil.infradead.org; Mon, 13 Nov 2017 09:34:31 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=+pW6Fj14pxHfuQjslC/bTCmd4YoNN6xMKV5k6UCd61I=; b=d3PebOQemmZjBfNjHUZ+9p/ai IfneTPzlgy49fbfBV5U1hbfyw/qXpd35LOh5RTIrUlZg/Rwym2zxDCBaGWXuvIC+gkwchl/XxMMVT NFGNWSIicCDgtofTMhmFzukC4dEZ/zATXD5KNQeLIqGvppriZTYc9QQ0Li0ytSyyzWFCxyHRog9to KNtRwtuPKj2H+XZhvDftCqCkB7YaHfBBIknWkYBFSJppGR3RO8XBO7YxUjbZ9uVh5/gRXbkoygWfY ctBjftMNXTtNpUqDl/xJ2bNEsUkejwCzhVN9eoqtvcrc8a6nLifoTOxa21tj3RQKnj/Lqko7ZNhri KJbWohrgw==; Received: from mail-wm0-x234.google.com ([2a00:1450:400c:c09::234]) by merlin.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1eEAtL-0005WO-I2 for linux-arm-kernel@lists.infradead.org; Mon, 13 Nov 2017 09:18:56 +0000 Received: by mail-wm0-x234.google.com with SMTP id b189so6525226wmd.5 for ; Mon, 13 Nov 2017 01:18:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=+pW6Fj14pxHfuQjslC/bTCmd4YoNN6xMKV5k6UCd61I=; b=eZ6++xvX/yYSVqjr4jmE6ss0yeIhs1GmKW/EAaA+EGEtS8aMCmadQP47vbPOZse5Xv 3paAl3ejpw1Uokti0a4T7V7W4vgdBNUp1bx7pzGoc+jE0An+en5XfPwNuC0G3HACK1IQ Xz2woKBOv5SBWwmn0tVXD00aY+qfKQRtId9gc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=+pW6Fj14pxHfuQjslC/bTCmd4YoNN6xMKV5k6UCd61I=; b=cu60dngC3SQ4/5BAQI1rze2WGfP0yJ2R5XieIr4cWgsowLsfmdb6SVbnax+UhpnqeO uqWWe5WBMLoeiKLfr3v7vwoba/gUOUMhzC6Rcb3igLIen2sxZBjXUC+6LtQlRVUCyhol N2FWC/QYCIx3bqegw7aUUsvQIOcqwioAPKcypla4aQhzEBtvKfSlFWqecYvYMCm2OWmK eB8UVEYUX2QSD3/XoJF8LtyklD/UxhVSm9wX/WrSF3nIGLXU8M2NiSlhCrOD2A/pgb+c iJgM1kzTZNQJR+p01SjnLcaFf9tGWDMgvwmh8ko3H0RtHZO8u/WpKOFK4zjydOPhsgrl HEBw== X-Gm-Message-State: AJaThX6A98V036XO2ovUBqh2X0F/D52oMjHYwqLcnwkQmSBqH2HGmM5h gDgM07NTis3eVy7tHjgxnwaSKQ== X-Google-Smtp-Source: AGs4zMYvGxedIOeS+c2YK5KekM89qE20tZJhZ2rFyT6dAso/eq7YacxKBbM1OcxeRmQzcdYwYRyPrg== X-Received: by 10.28.145.199 with SMTP id t190mr3580912wmd.40.1510564713416; Mon, 13 Nov 2017 01:18:33 -0800 (PST) Received: from localhost.localdomain (xd93dd96b.cust.hiper.dk. [217.61.217.107]) by smtp.gmail.com with ESMTPSA id x63sm9651399wma.39.2017.11.13.01.18.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 13 Nov 2017 01:18:32 -0800 (PST) From: Christoffer Dall To: Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= Subject: [PULL 25/27] KVM: arm/arm64: GICv4: Theory of operations Date: Mon, 13 Nov 2017 10:17:50 +0100 Message-Id: <20171113091752.10663-26-christoffer.dall@linaro.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171113091752.10663-1-christoffer.dall@linaro.org> References: <20171113091752.10663-1-christoffer.dall@linaro.org> X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marc Zyngier , Christoffer Dall , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Marc Zyngier Yet another braindump so I can free some cells... Acked-by: Christoffer Dall Signed-off-by: Marc Zyngier Signed-off-by: Christoffer Dall --- virt/kvm/arm/vgic/vgic-v4.c | 67 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c index bf874dd01fc0..915d09dc2638 100644 --- a/virt/kvm/arm/vgic/vgic-v4.c +++ b/virt/kvm/arm/vgic/vgic-v4.c @@ -23,6 +23,73 @@ #include "vgic.h" +/* + * How KVM uses GICv4 (insert rude comments here): + * + * The vgic-v4 layer acts as a bridge between several entities: + * - The GICv4 ITS representation offered by the ITS driver + * - VFIO, which is in charge of the PCI endpoint + * - The virtual ITS, which is the only thing the guest sees + * + * The configuration of VLPIs is triggered by a callback from VFIO, + * instructing KVM that a PCI device has been configured to deliver + * MSIs to a vITS. + * + * kvm_vgic_v4_set_forwarding() is thus called with the routing entry, + * and this is used to find the corresponding vITS data structures + * (ITS instance, device, event and irq) using a process that is + * extremely similar to the injection of an MSI. + * + * At this stage, we can link the guest's view of an LPI (uniquely + * identified by the routing entry) and the host irq, using the GICv4 + * driver mapping operation. Should the mapping succeed, we've then + * successfully upgraded the guest's LPI to a VLPI. We can then start + * with updating GICv4's view of the property table and generating an + * INValidation in order to kickstart the delivery of this VLPI to the + * guest directly, without software intervention. Well, almost. + * + * When the PCI endpoint is deconfigured, this operation is reversed + * with VFIO calling kvm_vgic_v4_unset_forwarding(). + * + * Once the VLPI has been mapped, it needs to follow any change the + * guest performs on its LPI through the vITS. For that, a number of + * command handlers have hooks to communicate these changes to the HW: + * - Any invalidation triggers a call to its_prop_update_vlpi() + * - The INT command results in a irq_set_irqchip_state(), which + * generates an INT on the corresponding VLPI. + * - The CLEAR command results in a irq_set_irqchip_state(), which + * generates an CLEAR on the corresponding VLPI. + * - DISCARD translates into an unmap, similar to a call to + * kvm_vgic_v4_unset_forwarding(). + * - MOVI is translated by an update of the existing mapping, changing + * the target vcpu, resulting in a VMOVI being generated. + * - MOVALL is translated by a string of mapping updates (similar to + * the handling of MOVI). MOVALL is horrible. + * + * Note that a DISCARD/MAPTI sequence emitted from the guest without + * reprogramming the PCI endpoint after MAPTI does not result in a + * VLPI being mapped, as there is no callback from VFIO (the guest + * will get the interrupt via the normal SW injection). Fixing this is + * not trivial, and requires some horrible messing with the VFIO + * internals. Not fun. Don't do that. + * + * Then there is the scheduling. Each time a vcpu is about to run on a + * physical CPU, KVM must tell the corresponding redistributor about + * it. And if we've migrated our vcpu from one CPU to another, we must + * tell the ITS (so that the messages reach the right redistributor). + * This is done in two steps: first issue a irq_set_affinity() on the + * irq corresponding to the vcpu, then call its_schedule_vpe(). You + * must be in a non-preemptible context. On exit, another call to + * its_schedule_vpe() tells the redistributor that we're done with the + * vcpu. + * + * Finally, the doorbell handling: Each vcpu is allocated an interrupt + * which will fire each time a VLPI is made pending whilst the vcpu is + * not running. Each time the vcpu gets blocked, the doorbell + * interrupt gets enabled. When the vcpu is unblocked (for whatever + * reason), the doorbell interrupt is disabled. + */ + #define DB_IRQ_FLAGS (IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING) static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)