From patchwork Mon Nov 13 09:17:50 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 10055333 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 058EE60586 for ; Mon, 13 Nov 2017 09:18:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0BC78291CC for ; Mon, 13 Nov 2017 09:18:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F40DE291B9; Mon, 13 Nov 2017 09:18:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 823FD291B9 for ; Mon, 13 Nov 2017 09:18:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752521AbdKMJSl (ORCPT ); Mon, 13 Nov 2017 04:18:41 -0500 Received: from mail-wm0-f52.google.com ([74.125.82.52]:35905 "EHLO mail-wm0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752436AbdKMJSe (ORCPT ); Mon, 13 Nov 2017 04:18:34 -0500 Received: by mail-wm0-f52.google.com with SMTP id r68so13287193wmr.1 for ; Mon, 13 Nov 2017 01:18:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=+pW6Fj14pxHfuQjslC/bTCmd4YoNN6xMKV5k6UCd61I=; b=eZ6++xvX/yYSVqjr4jmE6ss0yeIhs1GmKW/EAaA+EGEtS8aMCmadQP47vbPOZse5Xv 3paAl3ejpw1Uokti0a4T7V7W4vgdBNUp1bx7pzGoc+jE0An+en5XfPwNuC0G3HACK1IQ Xz2woKBOv5SBWwmn0tVXD00aY+qfKQRtId9gc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=+pW6Fj14pxHfuQjslC/bTCmd4YoNN6xMKV5k6UCd61I=; b=hsIgG0n1BBHgQW+BbSx6/sNF38U1SpzhPZN6bVeBnWyx/jsB5asoEwRAWHsEaFYoJV gOD6j/FV8cNIwR7T/RxtyBb0JiLupXH9g472zQPYlxDg0hB5QLWY3Zs0b5yu/SNewOhC 0pVttu2F7aHbTavOW9GAXyz8Ksp+YKY9O7HgPgCscWR8gBC28BkFE+YV0GeC48zaNBPy Pi8CLnVF7Y5/LWuCbI5hwIUR4xqaCAwZ7r+X323oiJuJWmuAcnkGzxI2dq/Rz+xNqfjd pol+5ftTTqd6pXNogkTS8uEUcxXh8gdLF+FwT5dkTm8d3uLw+AJbsT71qP+Vz0sS4hA1 WzCQ== X-Gm-Message-State: AJaThX49CHnaGluye8AxwTJPgVbddGtuxu7VJjCAGEZsom80YzuQNtBi /TN6iyCharrblCEzrtzR8vnpQw== X-Google-Smtp-Source: AGs4zMYvGxedIOeS+c2YK5KekM89qE20tZJhZ2rFyT6dAso/eq7YacxKBbM1OcxeRmQzcdYwYRyPrg== X-Received: by 10.28.145.199 with SMTP id t190mr3580912wmd.40.1510564713416; Mon, 13 Nov 2017 01:18:33 -0800 (PST) Received: from localhost.localdomain (xd93dd96b.cust.hiper.dk. [217.61.217.107]) by smtp.gmail.com with ESMTPSA id x63sm9651399wma.39.2017.11.13.01.18.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 13 Nov 2017 01:18:32 -0800 (PST) From: Christoffer Dall To: Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= Cc: Marc Zyngier , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Christoffer Dall Subject: [PULL 25/27] KVM: arm/arm64: GICv4: Theory of operations Date: Mon, 13 Nov 2017 10:17:50 +0100 Message-Id: <20171113091752.10663-26-christoffer.dall@linaro.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171113091752.10663-1-christoffer.dall@linaro.org> References: <20171113091752.10663-1-christoffer.dall@linaro.org> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Marc Zyngier Yet another braindump so I can free some cells... Acked-by: Christoffer Dall Signed-off-by: Marc Zyngier Signed-off-by: Christoffer Dall --- virt/kvm/arm/vgic/vgic-v4.c | 67 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/virt/kvm/arm/vgic/vgic-v4.c b/virt/kvm/arm/vgic/vgic-v4.c index bf874dd01fc0..915d09dc2638 100644 --- a/virt/kvm/arm/vgic/vgic-v4.c +++ b/virt/kvm/arm/vgic/vgic-v4.c @@ -23,6 +23,73 @@ #include "vgic.h" +/* + * How KVM uses GICv4 (insert rude comments here): + * + * The vgic-v4 layer acts as a bridge between several entities: + * - The GICv4 ITS representation offered by the ITS driver + * - VFIO, which is in charge of the PCI endpoint + * - The virtual ITS, which is the only thing the guest sees + * + * The configuration of VLPIs is triggered by a callback from VFIO, + * instructing KVM that a PCI device has been configured to deliver + * MSIs to a vITS. + * + * kvm_vgic_v4_set_forwarding() is thus called with the routing entry, + * and this is used to find the corresponding vITS data structures + * (ITS instance, device, event and irq) using a process that is + * extremely similar to the injection of an MSI. + * + * At this stage, we can link the guest's view of an LPI (uniquely + * identified by the routing entry) and the host irq, using the GICv4 + * driver mapping operation. Should the mapping succeed, we've then + * successfully upgraded the guest's LPI to a VLPI. We can then start + * with updating GICv4's view of the property table and generating an + * INValidation in order to kickstart the delivery of this VLPI to the + * guest directly, without software intervention. Well, almost. + * + * When the PCI endpoint is deconfigured, this operation is reversed + * with VFIO calling kvm_vgic_v4_unset_forwarding(). + * + * Once the VLPI has been mapped, it needs to follow any change the + * guest performs on its LPI through the vITS. For that, a number of + * command handlers have hooks to communicate these changes to the HW: + * - Any invalidation triggers a call to its_prop_update_vlpi() + * - The INT command results in a irq_set_irqchip_state(), which + * generates an INT on the corresponding VLPI. + * - The CLEAR command results in a irq_set_irqchip_state(), which + * generates an CLEAR on the corresponding VLPI. + * - DISCARD translates into an unmap, similar to a call to + * kvm_vgic_v4_unset_forwarding(). + * - MOVI is translated by an update of the existing mapping, changing + * the target vcpu, resulting in a VMOVI being generated. + * - MOVALL is translated by a string of mapping updates (similar to + * the handling of MOVI). MOVALL is horrible. + * + * Note that a DISCARD/MAPTI sequence emitted from the guest without + * reprogramming the PCI endpoint after MAPTI does not result in a + * VLPI being mapped, as there is no callback from VFIO (the guest + * will get the interrupt via the normal SW injection). Fixing this is + * not trivial, and requires some horrible messing with the VFIO + * internals. Not fun. Don't do that. + * + * Then there is the scheduling. Each time a vcpu is about to run on a + * physical CPU, KVM must tell the corresponding redistributor about + * it. And if we've migrated our vcpu from one CPU to another, we must + * tell the ITS (so that the messages reach the right redistributor). + * This is done in two steps: first issue a irq_set_affinity() on the + * irq corresponding to the vcpu, then call its_schedule_vpe(). You + * must be in a non-preemptible context. On exit, another call to + * its_schedule_vpe() tells the redistributor that we're done with the + * vcpu. + * + * Finally, the doorbell handling: Each vcpu is allocated an interrupt + * which will fire each time a VLPI is made pending whilst the vcpu is + * not running. Each time the vcpu gets blocked, the doorbell + * interrupt gets enabled. When the vcpu is unblocked (for whatever + * reason), the doorbell interrupt is disabled. + */ + #define DB_IRQ_FLAGS (IRQ_NOAUTOEN | IRQ_DISABLE_UNLAZY | IRQ_NO_BALANCING) static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)