From patchwork Tue Jan 8 06:03:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yan Zhao X-Patchwork-Id: 10751471 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9FFC213B4 for ; Tue, 8 Jan 2019 06:12:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8D9F328AF5 for ; Tue, 8 Jan 2019 06:12:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8176928B30; Tue, 8 Jan 2019 06:12:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EF6D428AF5 for ; Tue, 8 Jan 2019 06:12:18 +0000 (UTC) Received: from localhost ([127.0.0.1]:50982 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ggkcc-0001jK-8P for patchwork-qemu-devel@patchwork.kernel.org; Tue, 08 Jan 2019 01:12:18 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48579) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ggkbk-0001Cs-MV for qemu-devel@nongnu.org; Tue, 08 Jan 2019 01:11:25 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ggkbg-0003Ue-P4 for qemu-devel@nongnu.org; Tue, 08 Jan 2019 01:11:23 -0500 Received: from mga03.intel.com ([134.134.136.65]:5703) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ggkbf-0003Ko-Pe for qemu-devel@nongnu.org; Tue, 08 Jan 2019 01:11:20 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 Jan 2019 22:11:15 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,453,1539673200"; d="scan'208";a="104817672" Received: from joy-optiplex-7040.sh.intel.com ([10.239.13.9]) by orsmga007.jf.intel.com with ESMTP; 07 Jan 2019 22:11:12 -0800 From: Zhao Yan To: qemu-devel@nongnu.org Date: Tue, 8 Jan 2019 01:03:47 -0500 Message-Id: <20190108060348.3359-1-yan.y.zhao@intel.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 134.134.136.65 Subject: [Qemu-devel] [PATCH] vfio: assign idstr for VFIO's mmaped regions for migration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.williamson@redhat.com, Zhao Yan Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP if multiple regions in vfio are mmaped, their corresponding ramblocks are like below, i.e. their idstrs are "". (qemu) info ramblock Block Name PSize Offset Used Total pc.ram 4 KiB 0x0000000000000000 0x0000000020000000 0x0000000020000000 4 KiB 0x0000000021100000 0x0000000020000000 0x0000000020000000 4 KiB 0x0000000020900000 0x0000000000800000 0x0000000000800000 4 KiB 0x0000000020240000 0x0000000000687000 0x0000000000687000 4 KiB 0x00000000200c0000 0x0000000000178000 0x0000000000178000 pc.bios 4 KiB 0x0000000020000000 0x0000000000040000 0x0000000000040000 pc.rom 4 KiB 0x0000000020040000 0x0000000000020000 0x0000000000020000 This is because ramblocks' idstr are assigned by calling vmstate_register_ram(), but memory region of type ram device ptr does not call vmstate_register_ram(). vfio_region_mmap |->memory_region_init_ram_device_ptr |-> memory_region_init_ram_ptr Without empty idstrs will cause problem to snapshot copying during migration, because it uses ramblocks' idstr to identify ramblocks. ram_save_setup { … RAMBLOCK_FOREACH(block) { qemu_put_byte(f, strlen(block->idstr)); qemu_put_buffer(f, (uint8_t *)block->idstr,strlen(block->idstr)); qemu_put_be64(f, block->used_length); } … } ram_load() { block = qemu_ram_block_by_name(id); if (block) { if (length != block->used_length) { qemu_ram_resize(block, length, &local_err); } …. } } Therefore, in this patch, vmstate_register_ram() is called for memory region of type ram ptr, also a unique vfioid is assigned to vfio devices across source and target vms. e.g. in source vm, use qemu parameter -device vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:00:02.0/ 882cc4da-dede-11e7-9180-078a62063ab1,vfioid=igd and in target vm, use qemu paramter -device vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:00:02.0/ 5ac1fb20-2bbf-4842-bb7e-36c58c3be9cd,vfioid=igd Signed-off-by: Zhao Yan --- hw/vfio/pci.c | 8 +++++++- include/hw/vfio/vfio-common.h | 1 + memory.c | 4 ++++ 3 files changed, 12 insertions(+), 1 deletion(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index c0cb1ec289..7bc2ed0752 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -2533,7 +2533,12 @@ static void vfio_populate_device(VFIOPCIDevice *vdev, Error **errp) } for (i = VFIO_PCI_BAR0_REGION_INDEX; i < VFIO_PCI_ROM_REGION_INDEX; i++) { - char *name = g_strdup_printf("%s BAR %d", vbasedev->name, i); + char *name; + if (vbasedev->vfioid) { + name = g_strdup_printf("%s BAR %d", vbasedev->vfioid, i); + } else { + name = g_strdup_printf("%s BAR %d", vbasedev->name, i); + } ret = vfio_region_setup(OBJECT(vdev), vbasedev, &vdev->bars[i].region, i, name); @@ -3180,6 +3185,7 @@ static void vfio_instance_init(Object *obj) static Property vfio_pci_dev_properties[] = { DEFINE_PROP_PCI_HOST_DEVADDR("host", VFIOPCIDevice, host), DEFINE_PROP_STRING("sysfsdev", VFIOPCIDevice, vbasedev.sysfsdev), + DEFINE_PROP_STRING("vfioid", VFIOPCIDevice, vbasedev.vfioid), DEFINE_PROP_ON_OFF_AUTO("display", VFIOPCIDevice, display, ON_OFF_AUTO_OFF), DEFINE_PROP_UINT32("x-intx-mmap-timeout-ms", VFIOPCIDevice, diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 1b434d02f6..84bab94f52 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -108,6 +108,7 @@ typedef struct VFIODevice { struct VFIOGroup *group; char *sysfsdev; char *name; + char *vfioid; DeviceState *dev; int fd; int type; diff --git a/memory.c b/memory.c index d14c6dec1d..dbb29fa989 100644 --- a/memory.c +++ b/memory.c @@ -1588,6 +1588,7 @@ void memory_region_init_ram_ptr(MemoryRegion *mr, uint64_t size, void *ptr) { + DeviceState *owner_dev; memory_region_init(mr, owner, name, size); mr->ram = true; mr->terminates = true; @@ -1597,6 +1598,9 @@ void memory_region_init_ram_ptr(MemoryRegion *mr, /* qemu_ram_alloc_from_ptr cannot fail with ptr != NULL. */ assert(ptr != NULL); mr->ram_block = qemu_ram_alloc_from_ptr(size, ptr, mr, &error_fatal); + + owner_dev = DEVICE(owner); + vmstate_register_ram(mr, owner_dev); } void memory_region_init_ram_device_ptr(MemoryRegion *mr,