From patchwork Tue Jan 17 08:56:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cornelia Huck X-Patchwork-Id: 9520207 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3577C6020A for ; Tue, 17 Jan 2017 09:09:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2514F28515 for ; Tue, 17 Jan 2017 09:09:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 185A628517; Tue, 17 Jan 2017 09:09:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6F3B628515 for ; Tue, 17 Jan 2017 09:09:02 +0000 (UTC) Received: from localhost ([::1]:33669 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cTPlF-0003Aa-4F for patchwork-qemu-devel@patchwork.kernel.org; Tue, 17 Jan 2017 04:09:01 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48905) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cTPZe-0002A7-KU for qemu-devel@nongnu.org; Tue, 17 Jan 2017 03:57:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cTPZb-00084m-G3 for qemu-devel@nongnu.org; Tue, 17 Jan 2017 03:57:02 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:52085 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cTPZb-00084W-Bj for qemu-devel@nongnu.org; Tue, 17 Jan 2017 03:56:59 -0500 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v0H8tntr138200 for ; Tue, 17 Jan 2017 03:56:58 -0500 Received: from e06smtp14.uk.ibm.com (e06smtp14.uk.ibm.com [195.75.94.110]) by mx0b-001b2d01.pphosted.com with ESMTP id 281eucb1f5-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 17 Jan 2017 03:56:58 -0500 Received: from localhost by e06smtp14.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 17 Jan 2017 08:56:56 -0000 Received: from d06dlp01.portsmouth.uk.ibm.com (9.149.20.13) by e06smtp14.uk.ibm.com (192.168.101.144) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 17 Jan 2017 08:56:54 -0000 Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by d06dlp01.portsmouth.uk.ibm.com (Postfix) with ESMTP id 774DC17D805F; Tue, 17 Jan 2017 08:59:47 +0000 (GMT) Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v0H8urSs54132968; Tue, 17 Jan 2017 08:56:53 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2C3E552049; Tue, 17 Jan 2017 07:55:21 +0000 (GMT) Received: from gondolin.boeblingen.de.ibm.com (unknown [9.152.224.55]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTPS id EED8552041; Tue, 17 Jan 2017 07:55:20 +0000 (GMT) From: Cornelia Huck To: peter.maydell@linaro.org Date: Tue, 17 Jan 2017 09:56:38 +0100 X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170117085645.4267-1-cornelia.huck@de.ibm.com> References: <20170117085645.4267-1-cornelia.huck@de.ibm.com> X-TM-AS-GCONF: 00 X-Content-Scanned: Fidelis XPS MAILER x-cbid: 17011708-0016-0000-0000-00000428E59B X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17011708-0017-0000-0000-0000261B9D36 Message-Id: <20170117085645.4267-6-cornelia.huck@de.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-01-17_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=3 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1612050000 definitions=main-1701170122 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 Subject: [Qemu-devel] [PULL 05/12] s390x/pci: dynamically allocate iommu X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Yi Min Zhao , agraf@suse.de, qemu-devel@nongnu.org, borntraeger@de.ibm.com, jfrei@linux.vnet.ibm.com, Cornelia Huck Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Yi Min Zhao When initializing a PCI device, an address space is required during PCI core initialization and before the call to the embedding object hotplug callback. To provide this AS, we allocate a S390PCIIOMMU object containing this AS. Initialization of S390PCIIOMMU object is done before the PCI device is completely created. So that we cannot associate the IOMMU with the device at the moment. To track the IOMMU object, we use g_hash functions with the PCI device's bus address as a key to provide an array of pointers indexed by the PCI device's devfn to the allocated IOMMU. Signed-off-by: Yi Min Zhao Reviewed-by: Pierre Morel Signed-off-by: Cornelia Huck --- hw/s390x/s390-pci-bus.c | 73 +++++++++++++++++++++++++++++++++++++++---------- hw/s390x/s390-pci-bus.h | 7 ++++- 2 files changed, 65 insertions(+), 15 deletions(-) diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index 11244fc93c..04a4c3508e 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -432,11 +432,48 @@ static const MemoryRegionIOMMUOps s390_iommu_ops = { .translate = s390_translate_iommu, }; +static S390PCIIOMMU *s390_pci_get_iommu(S390pciState *s, PCIBus *bus, + int devfn) +{ + uint64_t key = (unsigned long)bus; + S390PCIIOMMUTable *table = g_hash_table_lookup(s->iommu_table, &key); + S390PCIIOMMU *iommu; + + if (!table) { + table = g_malloc0(sizeof(S390PCIIOMMUTable)); + table->key = key; + g_hash_table_insert(s->iommu_table, &table->key, table); + } + + iommu = table->iommu[PCI_SLOT(devfn)]; + if (!iommu) { + iommu = S390_PCI_IOMMU(object_new(TYPE_S390_PCI_IOMMU)); + + char *mr_name = g_strdup_printf("iommu-root-%02x:%02x.%01x", + pci_bus_num(bus), + PCI_SLOT(devfn), + PCI_FUNC(devfn)); + char *as_name = g_strdup_printf("iommu-pci-%02x:%02x.%01x", + pci_bus_num(bus), + PCI_SLOT(devfn), + PCI_FUNC(devfn)); + memory_region_init(&iommu->mr, OBJECT(iommu), mr_name, UINT64_MAX); + address_space_init(&iommu->as, &iommu->mr, as_name); + table->iommu[PCI_SLOT(devfn)] = iommu; + + g_free(mr_name); + g_free(as_name); + } + + return iommu; +} + static AddressSpace *s390_pci_dma_iommu(PCIBus *bus, void *opaque, int devfn) { S390pciState *s = opaque; + S390PCIIOMMU *iommu = s390_pci_get_iommu(s, bus, devfn); - return &s->iommu[PCI_SLOT(devfn)]->as; + return &iommu->as; } static uint8_t set_ind_atomic(uint64_t ind_loc, uint8_t to_be_set) @@ -520,19 +557,22 @@ void s390_pci_iommu_disable(S390PCIIOMMU *iommu) object_unparent(OBJECT(&iommu->iommu_mr)); } -static void s390_pcihost_init_as(S390pciState *s) +static void s390_pci_iommu_free(PCIBus *bus, int32_t devfn) { - int i; - S390PCIIOMMU *iommu; - - for (i = 0; i < PCI_SLOT_MAX; i++) { - iommu = g_malloc0(sizeof(S390PCIIOMMU)); - memory_region_init(&iommu->mr, OBJECT(s), - "iommu-root-s390", UINT64_MAX); - address_space_init(&iommu->as, &iommu->mr, "iommu-pci"); + uint64_t key = (unsigned long)bus; + S390pciState *s = s390_get_phb(); + S390PCIIOMMUTable *table = g_hash_table_lookup(s->iommu_table, &key); + S390PCIIOMMU *iommu = table ? table->iommu[PCI_SLOT(devfn)] : NULL; - s->iommu[i] = iommu; + if (!table || !iommu) { + return; } + + table->iommu[PCI_SLOT(devfn)] = NULL; + address_space_destroy(&iommu->as); + object_unparent(OBJECT(&iommu->mr)); + object_unparent(OBJECT(iommu)); + object_unref(OBJECT(iommu)); } static int s390_pcihost_init(SysBusDevice *dev) @@ -548,7 +588,6 @@ static int s390_pcihost_init(SysBusDevice *dev) s390_pci_set_irq, s390_pci_map_irq, NULL, get_system_memory(), get_system_io(), 0, 64, TYPE_PCI_BUS); - s390_pcihost_init_as(s); pci_setup_iommu(b, s390_pci_dma_iommu, s); bus = BUS(b); @@ -558,6 +597,8 @@ static int s390_pcihost_init(SysBusDevice *dev) s->bus = S390_PCI_BUS(qbus_create(TYPE_S390_PCI_BUS, DEVICE(s), NULL)); qbus_set_hotplug_handler(BUS(s->bus), DEVICE(s), NULL); + s->iommu_table = g_hash_table_new_full(g_int64_hash, g_int64_equal, + NULL, g_free); QTAILQ_INIT(&s->pending_sei); return 0; } @@ -661,7 +702,7 @@ static void s390_pcihost_hot_plug(HotplugHandler *hotplug_dev, } pbdev->pdev = pdev; - pbdev->iommu = s->iommu[PCI_SLOT(pdev->devfn)]; + pbdev->iommu = s390_pci_get_iommu(s, pdev->bus, pdev->devfn); pbdev->iommu->pbdev = pbdev; pbdev->state = ZPCI_FS_STANDBY; @@ -708,8 +749,9 @@ static void s390_pcihost_timer_cb(void *opaque) static void s390_pcihost_hot_unplug(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp) { - int i; PCIDevice *pci_dev = NULL; + PCIBus *bus; + int32_t devfn, i; S390PCIBusDevice *pbdev = NULL; S390pciState *s = s390_get_phb(); @@ -752,8 +794,11 @@ static void s390_pcihost_hot_unplug(HotplugHandler *hotplug_dev, s390_pci_generate_plug_event(HP_EVENT_STANDBY_TO_RESERVED, pbdev->fh, pbdev->fid); + bus = pci_dev->bus; + devfn = pci_dev->devfn; object_unparent(OBJECT(pci_dev)); s390_pci_msix_free(pbdev); + s390_pci_iommu_free(bus, devfn); pbdev->pdev = NULL; pbdev->state = ZPCI_FS_RESERVED; out: diff --git a/hw/s390x/s390-pci-bus.h b/hw/s390x/s390-pci-bus.h index 1d05c1a92e..fe108e9330 100644 --- a/hw/s390x/s390-pci-bus.h +++ b/hw/s390x/s390-pci-bus.h @@ -274,6 +274,11 @@ typedef struct S390PCIIOMMU { uint64_t pal; } S390PCIIOMMU; +typedef struct S390PCIIOMMUTable { + uint64_t key; + S390PCIIOMMU *iommu[PCI_SLOT_MAX]; +} S390PCIIOMMUTable; + typedef struct S390PCIBusDevice { DeviceState qdev; PCIDevice *pdev; @@ -304,7 +309,7 @@ typedef struct S390pciState { PCIHostState parent_obj; S390PCIBus *bus; S390PCIBusDevice *pbdev[PCI_SLOT_MAX]; - S390PCIIOMMU *iommu[PCI_SLOT_MAX]; + GHashTable *iommu_table; QTAILQ_HEAD(, SeiContainer) pending_sei; } S390pciState;