From patchwork Wed Jun 1 08:57:33 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Kardashevskiy X-Patchwork-Id: 9146767 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1143160777 for ; Wed, 1 Jun 2016 09:06:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0490E1FF65 for ; Wed, 1 Jun 2016 09:06:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ED9D922473; Wed, 1 Jun 2016 09:06:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00, MSGID_FROM_MTA_HEADER,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1C3401FF65 for ; Wed, 1 Jun 2016 09:06:20 +0000 (UTC) Received: from localhost ([::1]:41106 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b826V-0003Lr-2T for patchwork-qemu-devel@patchwork.kernel.org; Wed, 01 Jun 2016 05:06:19 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48907) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b821g-0007o2-Bi for qemu-devel@nongnu.org; Wed, 01 Jun 2016 05:01:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b821X-0001YN-4D for qemu-devel@nongnu.org; Wed, 01 Jun 2016 05:01:19 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:40721 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b821W-0001YI-W8 for qemu-devel@nongnu.org; Wed, 01 Jun 2016 05:01:11 -0400 Received: from pps.filterd (m0075420.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.11/8.16.0.11) with SMTP id u518wvs8046233 for ; Wed, 1 Jun 2016 05:01:10 -0400 Message-Id: <201606010901.u518wvs8046233@mx0a-001b2d01.pphosted.com> Received: from e23smtp05.au.ibm.com (e23smtp05.au.ibm.com [202.81.31.147]) by mx0a-001b2d01.pphosted.com with ESMTP id 239tadm83f-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Wed, 01 Jun 2016 05:01:10 -0400 Received: from localhost by e23smtp05.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 1 Jun 2016 19:01:06 +1000 Received: from d23dlp02.au.ibm.com (202.81.31.213) by e23smtp05.au.ibm.com (202.81.31.211) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 1 Jun 2016 19:01:04 +1000 X-IBM-Helo: d23dlp02.au.ibm.com X-IBM-MailFrom: aik@ozlabs.ru X-IBM-RcptTo: qemu-devel@nongnu.org;qemu-ppc@nongnu.org Received: from d23relay08.au.ibm.com (d23relay08.au.ibm.com [9.185.71.33]) by d23dlp02.au.ibm.com (Postfix) with ESMTP id 3FC912BB00AB; Wed, 1 Jun 2016 18:59:44 +1000 (EST) Received: from d23av04.au.ibm.com (d23av04.au.ibm.com [9.190.235.139]) by d23relay08.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id u518xbYG37421208; Wed, 1 Jun 2016 18:59:37 +1000 Received: from d23av04.au.ibm.com (localhost [127.0.0.1]) by d23av04.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id u518x0VS021180; Wed, 1 Jun 2016 18:59:00 +1000 Received: from ozlabs.au.ibm.com (ozlabs.au.ibm.com [9.192.253.14]) by d23av04.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id u518wxC8020328; Wed, 1 Jun 2016 18:58:59 +1000 Received: from bran.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) by ozlabs.au.ibm.com (Postfix) with ESMTP id 8CEA6A0132; Wed, 1 Jun 2016 18:57:49 +1000 (AEST) Received: from vpl2.ozlabs.ibm.com (vpl2.ozlabs.ibm.com [10.61.141.27]) by bran.ozlabs.ibm.com (Postfix) with ESMTP id 576CDE3AE3; Wed, 1 Jun 2016 18:57:49 +1000 (AEST) From: Alexey Kardashevskiy To: qemu-devel@nongnu.org Date: Wed, 1 Jun 2016 18:57:33 +1000 X-Mailer: git-send-email 2.5.0.rc3 In-Reply-To: <1464771463-37214-1-git-send-email-aik@ozlabs.ru> References: <1464771463-37214-1-git-send-email-aik@ozlabs.ru> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16060109-0016-0000-0000-000001A0A084 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16060109-0017-0000-0000-000004BE7DD0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-06-01_03:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=98 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1604210000 definitions=main-1606010107 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-Received-From: 148.163.158.5 Subject: [Qemu-devel] [PATCH qemu v17 02/12] spapr_iommu: Introduce "enabled" state for TCE table X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexey Kardashevskiy , Alex Williamson , qemu-ppc@nongnu.org, Alexander Graf , David Gibson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Currently TCE tables are created once at start and their sizes never change. We are going to change that by introducing a Dynamic DMA windows support where DMA configuration may change during the guest execution. This changes spapr_tce_new_table() to create an empty zero-size IOMMU memory region (IOMMU MR). Only LIOBN is assigned by the time of creation. It still will be called once at the owner object (VIO or PHB) creation. This introduces an "enabled" state for TCE table objects, some helper functions are added: - spapr_tce_table_enable() receives TCE table parameters, stores in sPAPRTCETable and allocates a guest view of the TCE table (in the user space or KVM) and sets the correct size on the IOMMU MR; - spapr_tce_table_disable() disposes the table and resets the IOMMU MR size; it is made public as the following DDW code will be using it. This changes the PHB reset handler to do the default DMA initialization instead of spapr_phb_realize(). This does not make differenct now but later with more than just one DMA window, we will have to remove them all and create the default one on a system reset. No visible change in behaviour is expected except the actual table will be reallocated every reset. We might optimize this later. The other way to implement this would be dynamically create/remove the TCE table QOM objects but this would make migration impossible as the migration code expects all QOM objects to exist at the receiver so we have to have TCE table objects created when migration begins. Signed-off-by: Alexey Kardashevskiy --- Changes: v17: * spapr_tce_table_unrealize() calls spapr_tce_table_do_disable() directly * moved spapr_tce_table_disable() to next patch as it is not used here * removed @enabled as nb_table indicates already if the table is enabled v15: * made adjustments after removing spapr_phb_dma_window_enable() v14: * added spapr_tce_table_do_disable(), will make difference in following patch with fully dynamic table migration --- hw/ppc/spapr_iommu.c | 68 ++++++++++++++++++++++++++++++++------------------ hw/ppc/spapr_pci.c | 8 +++--- hw/ppc/spapr_vio.c | 8 +++--- include/hw/ppc/spapr.h | 9 +++---- 4 files changed, 56 insertions(+), 37 deletions(-) diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c index 96bb018..de63467 100644 --- a/hw/ppc/spapr_iommu.c +++ b/hw/ppc/spapr_iommu.c @@ -17,6 +17,7 @@ * License along with this library; if not, see . */ #include "qemu/osdep.h" +#include "qemu/error-report.h" #include "hw/hw.h" #include "qemu/log.h" #include "sysemu/kvm.h" @@ -175,15 +176,9 @@ static int spapr_tce_table_realize(DeviceState *dev) sPAPRTCETable *tcet = SPAPR_TCE_TABLE(dev); tcet->fd = -1; - tcet->table = spapr_tce_alloc_table(tcet->liobn, - tcet->page_shift, - tcet->nb_table, - &tcet->fd, - tcet->need_vfio); - + tcet->need_vfio = false; memory_region_init_iommu(&tcet->iommu, OBJECT(dev), &spapr_iommu_ops, - "iommu-spapr", - (uint64_t)tcet->nb_table << tcet->page_shift); + "iommu-spapr", 0); QLIST_INSERT_HEAD(&spapr_tce_tables, tcet, list); @@ -225,14 +220,10 @@ void spapr_tce_set_need_vfio(sPAPRTCETable *tcet, bool need_vfio) tcet->table = newtable; } -sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn, - uint64_t bus_offset, - uint32_t page_shift, - uint32_t nb_table, - bool need_vfio) +sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn) { sPAPRTCETable *tcet; - char tmp[64]; + char tmp[32]; if (spapr_tce_find_by_liobn(liobn)) { fprintf(stderr, "Attempted to create TCE table with duplicate" @@ -240,16 +231,8 @@ sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn, return NULL; } - if (!nb_table) { - return NULL; - } - tcet = SPAPR_TCE_TABLE(object_new(TYPE_SPAPR_TCE_TABLE)); tcet->liobn = liobn; - tcet->bus_offset = bus_offset; - tcet->page_shift = page_shift; - tcet->nb_table = nb_table; - tcet->need_vfio = need_vfio; snprintf(tmp, sizeof(tmp), "tce-table-%x", liobn); object_property_add_child(OBJECT(owner), tmp, OBJECT(tcet), NULL); @@ -259,14 +242,51 @@ sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn, return tcet; } +void spapr_tce_table_enable(sPAPRTCETable *tcet, + uint32_t page_shift, uint64_t bus_offset, + uint32_t nb_table) +{ + if (tcet->nb_table) { + error_report("Warning: trying to enable already enabled TCE table"); + return; + } + + tcet->bus_offset = bus_offset; + tcet->page_shift = page_shift; + tcet->nb_table = nb_table; + tcet->table = spapr_tce_alloc_table(tcet->liobn, + tcet->page_shift, + tcet->nb_table, + &tcet->fd, + tcet->need_vfio); + + memory_region_set_size(&tcet->iommu, + (uint64_t)tcet->nb_table << tcet->page_shift); +} + +static void spapr_tce_table_disable(sPAPRTCETable *tcet) +{ + if (!tcet->nb_table) { + return; + } + + memory_region_set_size(&tcet->iommu, 0); + + spapr_tce_free_table(tcet->table, tcet->fd, tcet->nb_table); + tcet->fd = -1; + tcet->table = NULL; + tcet->bus_offset = 0; + tcet->page_shift = 0; + tcet->nb_table = 0; +} + static void spapr_tce_table_unrealize(DeviceState *dev, Error **errp) { sPAPRTCETable *tcet = SPAPR_TCE_TABLE(dev); QLIST_REMOVE(tcet, list); - spapr_tce_free_table(tcet->table, tcet->fd, tcet->nb_table); - tcet->fd = -1; + spapr_tce_table_disable(tcet); } MemoryRegion *spapr_tce_get_iommu(sPAPRTCETable *tcet) diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c index 856aec7..7688ae0 100644 --- a/hw/ppc/spapr_pci.c +++ b/hw/ppc/spapr_pci.c @@ -1463,8 +1463,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp) } nb_table = sphb->dma_win_size >> SPAPR_TCE_PAGE_SHIFT; - tcet = spapr_tce_new_table(DEVICE(sphb), sphb->dma_liobn, - 0, SPAPR_TCE_PAGE_SHIFT, nb_table, false); + tcet = spapr_tce_new_table(DEVICE(sphb), sphb->dma_liobn); if (!tcet) { error_setg(errp, "Unable to create TCE table for %s", sphb->dtbusname); @@ -1472,7 +1471,10 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp) } /* Register default 32bit DMA window */ - memory_region_add_subregion(&sphb->iommu_root, sphb->dma_win_addr, + spapr_tce_table_enable(tcet, SPAPR_TCE_PAGE_SHIFT, sphb->dma_win_addr, + nb_table); + + memory_region_add_subregion(&sphb->iommu_root, tcet->bus_offset, spapr_tce_get_iommu(tcet)); sphb->msi = g_hash_table_new_full(g_int_hash, g_int_equal, g_free, g_free); diff --git a/hw/ppc/spapr_vio.c b/hw/ppc/spapr_vio.c index d084aed..3d9b9c6 100644 --- a/hw/ppc/spapr_vio.c +++ b/hw/ppc/spapr_vio.c @@ -483,11 +483,9 @@ static void spapr_vio_busdev_realize(DeviceState *qdev, Error **errp) memory_region_add_subregion_overlap(&dev->mrroot, 0, &dev->mrbypass, 1); address_space_init(&dev->as, &dev->mrroot, qdev->id); - dev->tcet = spapr_tce_new_table(qdev, liobn, - 0, - SPAPR_TCE_PAGE_SHIFT, - pc->rtce_window_size >> - SPAPR_TCE_PAGE_SHIFT, false); + dev->tcet = spapr_tce_new_table(qdev, liobn); + spapr_tce_table_enable(dev->tcet, SPAPR_TCE_PAGE_SHIFT, 0, + pc->rtce_window_size >> SPAPR_TCE_PAGE_SHIFT); dev->tcet->vdev = dev; memory_region_add_subregion_overlap(&dev->mrroot, 0, spapr_tce_get_iommu(dev->tcet), 2); diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index 815d5ee..26c327d 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -561,11 +561,10 @@ void spapr_events_fdt_skel(void *fdt, uint32_t epow_irq); int spapr_h_cas_compose_response(sPAPRMachineState *sm, target_ulong addr, target_ulong size, bool cpu_update, bool memory_update); -sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn, - uint64_t bus_offset, - uint32_t page_shift, - uint32_t nb_table, - bool need_vfio); +sPAPRTCETable *spapr_tce_new_table(DeviceState *owner, uint32_t liobn); +void spapr_tce_table_enable(sPAPRTCETable *tcet, + uint32_t page_shift, uint64_t bus_offset, + uint32_t nb_table); void spapr_tce_set_need_vfio(sPAPRTCETable *tcet, bool need_vfio); MemoryRegion *spapr_tce_get_iommu(sPAPRTCETable *tcet);