From patchwork Mon Apr 4 09:33:36 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Kardashevskiy X-Patchwork-Id: 8739251 Return-Path: X-Original-To: patchwork-qemu-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 54DD59F39A for ; Mon, 4 Apr 2016 09:41:34 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 89672201C0 for ; Mon, 4 Apr 2016 09:41:33 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6DCAB2017E for ; Mon, 4 Apr 2016 09:41:31 +0000 (UTC) Received: from localhost ([::1]:57567 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1an10k-0003Aw-PG for patchwork-qemu-devel@patchwork.kernel.org; Mon, 04 Apr 2016 05:41:30 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44718) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1an0uv-0000Nh-U1 for qemu-devel@nongnu.org; Mon, 04 Apr 2016 05:35:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1an0ut-0000Yu-Ne for qemu-devel@nongnu.org; Mon, 04 Apr 2016 05:35:29 -0400 Received: from e23smtp02.au.ibm.com ([202.81.31.144]:60905) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1an0us-0000Xj-VR for qemu-devel@nongnu.org; Mon, 04 Apr 2016 05:35:27 -0400 Received: from localhost by e23smtp02.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 4 Apr 2016 19:35:25 +1000 Received: from d23dlp02.au.ibm.com (202.81.31.213) by e23smtp02.au.ibm.com (202.81.31.208) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 4 Apr 2016 19:35:22 +1000 X-IBM-Helo: d23dlp02.au.ibm.com X-IBM-MailFrom: aik@ozlabs.ru X-IBM-RcptTo: qemu-devel@nongnu.org;qemu-ppc@nongnu.org Received: from d23relay06.au.ibm.com (d23relay06.au.ibm.com [9.185.63.219]) by d23dlp02.au.ibm.com (Postfix) with ESMTP id 6C3772BB0057; Mon, 4 Apr 2016 19:35:14 +1000 (EST) Received: from d23av04.au.ibm.com (d23av04.au.ibm.com [9.190.235.139]) by d23relay06.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id u349Z6Kt8257866; Mon, 4 Apr 2016 19:35:14 +1000 Received: from d23av04.au.ibm.com (localhost [127.0.0.1]) by d23av04.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id u349YfQV013635; Mon, 4 Apr 2016 19:34:42 +1000 Received: from ozlabs.au.ibm.com (ozlabs.au.ibm.com [9.192.253.14]) by d23av04.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id u349YfYS012594; Mon, 4 Apr 2016 19:34:41 +1000 Received: from bran.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) by ozlabs.au.ibm.com (Postfix) with ESMTP id 0214DA03B0; Mon, 4 Apr 2016 19:33:53 +1000 (AEST) Received: from vpl2.ozlabs.ibm.com (vpl2.ozlabs.ibm.com [10.61.141.27]) by bran.ozlabs.ibm.com (Postfix) with ESMTP id F1F30E3A0C; Mon, 4 Apr 2016 19:33:52 +1000 (AEST) From: Alexey Kardashevskiy To: qemu-devel@nongnu.org Date: Mon, 4 Apr 2016 19:33:36 +1000 Message-Id: <1459762426-18440-8-git-send-email-aik@ozlabs.ru> X-Mailer: git-send-email 2.5.0.rc3 In-Reply-To: <1459762426-18440-1-git-send-email-aik@ozlabs.ru> References: <1459762426-18440-1-git-send-email-aik@ozlabs.ru> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16040409-0005-0000-0000-0000073F25C2 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 202.81.31.144 Cc: Alexey Kardashevskiy , Alex Williamson , qemu-ppc@nongnu.org, Alexander Graf , David Gibson Subject: [Qemu-devel] [PATCH qemu v15 07/17] spapr_iommu: Migrate full state X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The source guest could have reallocated the default TCE table and migrate bigger/smaller table. This adds reallocation in post_load() if the default table size is different on source and destination. This adds @bus_offset, @page_shift, @enabled to the migration stream. These cannot change without dynamic DMA windows so no change in behavior is expected now. Signed-off-by: Alexey Kardashevskiy Reviewed-by: David Gibson --- Changes: v15: * squashed "migrate full state" into this * added missing tcet->mig_nb_table initialization in spapr_tce_table_pre_save() * instead of bumping the version, moved extra parameters to subsection v14: * new to the series --- hw/ppc/spapr_iommu.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++++-- include/hw/ppc/spapr.h | 2 ++ trace-events | 2 ++ 3 files changed, 69 insertions(+), 2 deletions(-) diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c index 9bcd3f6..52b1e0d 100644 --- a/hw/ppc/spapr_iommu.c +++ b/hw/ppc/spapr_iommu.c @@ -137,33 +137,96 @@ static IOMMUTLBEntry spapr_tce_translate_iommu(MemoryRegion *iommu, hwaddr addr, return ret; } +static void spapr_tce_table_pre_save(void *opaque) +{ + sPAPRTCETable *tcet = SPAPR_TCE_TABLE(opaque); + + tcet->mig_table = tcet->table; + tcet->mig_nb_table = tcet->nb_table; + + trace_spapr_iommu_pre_save(tcet->liobn, tcet->mig_nb_table, + tcet->bus_offset, tcet->page_shift); +} + +static void spapr_tce_table_do_enable(sPAPRTCETable *tcet); +static void spapr_tce_table_do_disable(sPAPRTCETable *tcet); + static int spapr_tce_table_post_load(void *opaque, int version_id) { sPAPRTCETable *tcet = SPAPR_TCE_TABLE(opaque); + uint32_t old_nb_table = tcet->nb_table; if (tcet->vdev) { spapr_vio_set_bypass(tcet->vdev, tcet->bypass); } + if (tcet->enabled) { + if (tcet->nb_table != tcet->mig_nb_table) { + if (tcet->nb_table) { + spapr_tce_table_do_disable(tcet); + } + tcet->nb_table = tcet->mig_nb_table; + spapr_tce_table_do_enable(tcet); + } + + memcpy(tcet->table, tcet->mig_table, + tcet->nb_table * sizeof(tcet->table[0])); + + free(tcet->mig_table); + tcet->mig_table = NULL; + } else if (tcet->table) { + /* Destination guest has a default table but source does not -> free */ + spapr_tce_table_do_disable(tcet); + } + + trace_spapr_iommu_post_load(tcet->liobn, old_nb_table, tcet->nb_table, + tcet->bus_offset, tcet->page_shift); + return 0; } +static bool spapr_tce_table_ex_needed(void *opaque) +{ + sPAPRTCETable *tcet = opaque; + + return tcet->bus_offset || tcet->page_shift != 0xC; +} + +static const VMStateDescription vmstate_spapr_tce_table_ex = { + .name = "spapr_iommu_ex", + .version_id = 1, + .minimum_version_id = 1, + .needed = spapr_tce_table_ex_needed, + .fields = (VMStateField[]) { + VMSTATE_BOOL(enabled, sPAPRTCETable), + VMSTATE_UINT64(bus_offset, sPAPRTCETable), + VMSTATE_UINT32(page_shift, sPAPRTCETable), + VMSTATE_END_OF_LIST() + }, +}; + static const VMStateDescription vmstate_spapr_tce_table = { .name = "spapr_iommu", .version_id = 2, .minimum_version_id = 2, + .pre_save = spapr_tce_table_pre_save, .post_load = spapr_tce_table_post_load, .fields = (VMStateField []) { /* Sanity check */ VMSTATE_UINT32_EQUAL(liobn, sPAPRTCETable), - VMSTATE_UINT32_EQUAL(nb_table, sPAPRTCETable), /* IOMMU state */ + VMSTATE_UINT32(mig_nb_table, sPAPRTCETable), VMSTATE_BOOL(bypass, sPAPRTCETable), - VMSTATE_VARRAY_UINT32(table, sPAPRTCETable, nb_table, 0, vmstate_info_uint64, uint64_t), + VMSTATE_VARRAY_UINT32_ALLOC(mig_table, sPAPRTCETable, mig_nb_table, 0, + vmstate_info_uint64, uint64_t), VMSTATE_END_OF_LIST() }, + .subsections = (const VMStateDescription*[]) { + &vmstate_spapr_tce_table_ex, + NULL + } }; static MemoryRegionIOMMUOps spapr_iommu_ops = { diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index 75b0b55..c1ea49c 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -545,6 +545,8 @@ struct sPAPRTCETable { uint64_t bus_offset; uint32_t page_shift; uint64_t *table; + uint32_t mig_nb_table; + uint64_t *mig_table; bool bypass; bool need_vfio; int fd; diff --git a/trace-events b/trace-events index 62dcbba..4335b9b 100644 --- a/trace-events +++ b/trace-events @@ -1431,6 +1431,8 @@ spapr_iommu_pci_indirect(uint64_t liobn, uint64_t ioba, uint64_t tce, uint64_t i spapr_iommu_pci_stuff(uint64_t liobn, uint64_t ioba, uint64_t tce_value, uint64_t npages, uint64_t ret) "liobn=%"PRIx64" ioba=0x%"PRIx64" tcevalue=0x%"PRIx64" npages=%"PRId64" ret=%"PRId64 spapr_iommu_xlate(uint64_t liobn, uint64_t ioba, uint64_t tce, unsigned perm, unsigned pgsize) "liobn=%"PRIx64" 0x%"PRIx64" -> 0x%"PRIx64" perm=%u mask=%x" spapr_iommu_new_table(uint64_t liobn, void *table, int fd) "liobn=%"PRIx64" table=%p fd=%d" +spapr_iommu_pre_save(uint64_t liobn, uint32_t nb, uint64_t offs, uint32_t ps) "liobn=%"PRIx64" %"PRIx32" bus_offset=%"PRIx64" ps=%"PRIu32 +spapr_iommu_post_load(uint64_t liobn, uint32_t pre_nb, uint32_t post_nb, uint64_t offs, uint32_t ps) "liobn=%"PRIx64" %"PRIx32" => %"PRIx32" bus_offset=%"PRIx64" ps=%"PRIu32 # hw/ppc/ppc.c ppc_tb_adjust(uint64_t offs1, uint64_t offs2, int64_t diff, int64_t seconds) "adjusted from 0x%"PRIx64" to 0x%"PRIx64", diff %"PRId64" (%"PRId64"s)"