From patchwork Thu Mar 1 10:33:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Yi L" X-Patchwork-Id: 10250979 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 343FF602B5 for ; Thu, 1 Mar 2018 11:10:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 24B5328FE8 for ; Thu, 1 Mar 2018 11:10:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 198AB28FEA; Thu, 1 Mar 2018 11:10:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4AC6228FFF for ; Thu, 1 Mar 2018 11:10:28 +0000 (UTC) Received: from localhost ([::1]:55638 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erM6V-0005Yi-9B for patchwork-qemu-devel@patchwork.kernel.org; Thu, 01 Mar 2018 06:10:27 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53158) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLnN-0005cb-4c for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:50:42 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erLnI-0000BJ-Bc for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:50:41 -0500 Received: from mga14.intel.com ([192.55.52.115]:7359) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1erLnI-00006Y-34 for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:50:36 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Mar 2018 02:50:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,408,1515484800"; d="scan'208";a="208048690" Received: from sky-dev.bj.intel.com ([10.238.144.105]) by fmsmga006.fm.intel.com with ESMTP; 01 Mar 2018 02:50:33 -0800 From: "Liu, Yi L" To: qemu-devel@nongnu.org, mst@redhat.com, david@gibson.dropbear.id.au Date: Thu, 1 Mar 2018 18:33:28 +0800 Message-Id: <1519900415-30314-6-git-send-email-yi.l.liu@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1519900415-30314-1-git-send-email-yi.l.liu@linux.intel.com> References: <1519900415-30314-1-git-send-email-yi.l.liu@linux.intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH v3 05/12] hw/pci: introduce PCISVAOps to PCIDevice X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Liu, Yi L" , kevin.tian@intel.com, yi.l.liu@intel.com, jasowang@redhat.com, peterx@redhat.com, alex.williamson@redhat.com, pbonzini@redhat.com, eric.auger.pro@gmail.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch intoduces PCISVAOps for virt-SVA. So far, to setup virt-SVA for assigned SVA capable device, needs to config host translation structures. e.g. for VT-d, needs to set the guest pasid table to host and enable nested translation. Besides, vIOMMU emulator needs to forward guest's cache invalidation to host. On VT-d, it is guest's invalidation to 1st level translation related cache, such invalidation should be forwarded to host. Proposed PCISVAOps are: * sva_bind_guest_pasid_table: set the guest pasid table to host, and enable nested translation in host * sva_register_notifier: register sva_notifier to forward guest's cache invalidation to host * sva_unregister_notifier: unregister sva_notifier The PCISVAOps should be provided by vfio or modules alike. Mainly for assigned SVA capable devices. Take virt-SVA on VT-d as an exmaple: If a guest wants to setup virt-SVA for an assigned SVA capable device, it programs its context entry. vIOMMU emulator captures guest's context entry programming, and figure out the target device. vIOMMU emulator use the pci_device_sva_bind_pasid_table() API to bind the guest pasid table to host. Guest would also program its pasid table. vIOMMU emulator captures guest's pasid entry programming. In Qemu, needs to allocate an AddressSpace to stand for the pasid tagged address space and Qemu also needs to register sva_notifier to forward future cache invalidation request to host. Allocating AddressSpace to stand for the pasid tagged address space is for the emulation of emulated SVA capable devices. Emulated SVA capable devices may issue SVA aware DMAs, Qemu needs to emulate read/write to a pasid tagged AddressSpace. Thus needs an abstraction for such address space in Qemu. Signed-off-by: Liu, Yi L --- hw/pci/pci.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++++ include/hw/pci/pci.h | 21 ++++++++++++++++++ 2 files changed, 81 insertions(+) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index e006b6a..157fe21 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -2573,6 +2573,66 @@ void pci_setup_iommu(PCIBus *bus, PCIIOMMUFunc fn, void *opaque) bus->iommu_opaque = opaque; } +void pci_setup_sva_ops(PCIDevice *dev, PCISVAOps *ops) +{ + if (dev) { + dev->sva_ops = ops; + } + return; +} + +void pci_device_sva_bind_pasid_table(PCIBus *bus, + int32_t devfn, uint64_t addr, uint32_t size) +{ + PCIDevice *dev; + + if (!bus) { + return; + } + + dev = bus->devices[devfn]; + if (dev && dev->sva_ops) { + dev->sva_ops->sva_bind_pasid_table(bus, devfn, addr, size); + } + return; +} + +void pci_device_sva_register_notifier(PCIBus *bus, int32_t devfn, + IOMMUSVAContext *sva_ctx) +{ + PCIDevice *dev; + + if (!bus) { + return; + } + + dev = bus->devices[devfn]; + if (dev && dev->sva_ops) { + dev->sva_ops->sva_register_notifier(bus, + devfn, + sva_ctx); + } + return; +} + +void pci_device_sva_unregister_notifier(PCIBus *bus, int32_t devfn, + IOMMUSVAContext *sva_ctx) +{ + PCIDevice *dev; + + if (!bus) { + return; + } + + dev = bus->devices[devfn]; + if (dev && dev->sva_ops) { + dev->sva_ops->sva_unregister_notifier(bus, + devfn, + sva_ctx); + } + return; +} + static void pci_dev_get_w64(PCIBus *b, PCIDevice *dev, void *opaque) { Range *range = opaque; diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index d8c18c7..32889a4 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -10,6 +10,8 @@ #include "hw/pci/pcie.h" +#include "hw/core/pasid.h" + extern bool pci_available; /* PCI bus */ @@ -262,6 +264,16 @@ struct PCIReqIDCache { }; typedef struct PCIReqIDCache PCIReqIDCache; +typedef struct PCISVAOps PCISVAOps; +struct PCISVAOps { + void (*sva_bind_pasid_table)(PCIBus *bus, int32_t devfn, + uint64_t pasidt_addr, uint32_t size); + void (*sva_register_notifier)(PCIBus *bus, int32_t devfn, + IOMMUSVAContext *sva_ctx); + void (*sva_unregister_notifier)(PCIBus *bus, int32_t devfn, + IOMMUSVAContext *sva_ctx); +}; + struct PCIDevice { DeviceState qdev; @@ -351,6 +363,7 @@ struct PCIDevice { MSIVectorUseNotifier msix_vector_use_notifier; MSIVectorReleaseNotifier msix_vector_release_notifier; MSIVectorPollNotifier msix_vector_poll_notifier; + PCISVAOps *sva_ops; }; void pci_register_bar(PCIDevice *pci_dev, int region_num, @@ -477,6 +490,14 @@ typedef AddressSpace *(*PCIIOMMUFunc)(PCIBus *, void *, int); AddressSpace *pci_device_iommu_address_space(PCIDevice *dev); void pci_setup_iommu(PCIBus *bus, PCIIOMMUFunc fn, void *opaque); +void pci_setup_sva_ops(PCIDevice *dev, PCISVAOps *ops); +void pci_device_sva_bind_pasid_table(PCIBus *bus, int32_t devfn, + uint64_t pasidt_addr, uint32_t size); +void pci_device_sva_register_notifier(PCIBus *bus, int32_t devfn, + IOMMUSVAContext *sva_ctx); +void pci_device_sva_unregister_notifier(PCIBus *bus, int32_t devfn, + IOMMUSVAContext *sva_ctx); + static inline void pci_set_byte(uint8_t *config, uint8_t val) {