From patchwork Fri Apr 9 03:20:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Rao, Lei" X-Patchwork-Id: 12192929 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A42FC433B4 for ; Fri, 9 Apr 2021 03:46:25 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BF52D61042 for ; Fri, 9 Apr 2021 03:46:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BF52D61042 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:46814 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lUi6B-0007tq-Tr for qemu-devel@archiver.kernel.org; Thu, 08 Apr 2021 23:46:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52038) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lUi2n-0003a4-CJ for qemu-devel@nongnu.org; Thu, 08 Apr 2021 23:42:53 -0400 Received: from mga05.intel.com ([192.55.52.43]:12420) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lUi2l-0007T4-L6 for qemu-devel@nongnu.org; Thu, 08 Apr 2021 23:42:53 -0400 IronPort-SDR: 6o8rLchXohsKCWSJkmEbkG8Oy7j93VxbujKB1A0qd8jJZxCvILwqYLrIr7j6f1wzj2alfYGT3X xTuHIi1cPA7Q== X-IronPort-AV: E=McAfee;i="6000,8403,9948"; a="278957356" X-IronPort-AV: E=Sophos;i="5.82,208,1613462400"; d="scan'208";a="278957356" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Apr 2021 20:42:51 -0700 IronPort-SDR: HE+EyU4nCCBPg0WgRjene2detCeELippOgYGuGHu0XTT8DLgEMZ2w4TjXQIzrFpwqcab1bHzuT uuiGCKvszi6Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,208,1613462400"; d="scan'208";a="416081143" Received: from unknown (HELO localhost.localdomain.bj.intel.com) ([10.240.192.103]) by fmsmga008.fm.intel.com with ESMTP; 08 Apr 2021 20:42:48 -0700 From: leirao To: chen.zhang@intel.com, lizhijian@cn.fujitsu.com, jasowang@redhat.com, quintela@redhat.com, dgilbert@redhat.com, pbonzini@redhat.com, lukasstraub2@web.de Subject: [PATCH v6 08/10] Reduce the PVM stop time during Checkpoint Date: Thu, 8 Apr 2021 23:20:54 -0400 Message-Id: <1617938456-315058-9-git-send-email-lei.rao@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1617938456-315058-1-git-send-email-lei.rao@intel.com> References: <1617938456-315058-1-git-send-email-lei.rao@intel.com> Received-SPF: pass client-ip=192.55.52.43; envelope-from=lei.rao@intel.com; helo=mga05.intel.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Rao, Lei" , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: "Rao, Lei" When flushing memory from ram cache to ram during every checkpoint on secondary VM, we can copy continuous chunks of memory instead of 4096 bytes per time to reduce the time of VM stop during checkpoint. Signed-off-by: Lei Rao Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Lukas Straub Tested-by: Lukas Straub --- migration/ram.c | 48 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 45 insertions(+), 3 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index f9d60f0..8661d82 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -822,6 +822,41 @@ unsigned long migration_bitmap_find_dirty(RAMState *rs, RAMBlock *rb, return next; } +/* + * colo_bitmap_find_diry:find contiguous dirty pages from start + * + * Returns the page offset within memory region of the start of the contiguout + * dirty page + * + * @rs: current RAM state + * @rb: RAMBlock where to search for dirty pages + * @start: page where we start the search + * @num: the number of contiguous dirty pages + */ +static inline +unsigned long colo_bitmap_find_dirty(RAMState *rs, RAMBlock *rb, + unsigned long start, unsigned long *num) +{ + unsigned long size = rb->used_length >> TARGET_PAGE_BITS; + unsigned long *bitmap = rb->bmap; + unsigned long first, next; + + *num = 0; + + if (ramblock_is_ignored(rb)) { + return size; + } + + first = find_next_bit(bitmap, size, start); + if (first >= size) { + return first; + } + next = find_next_zero_bit(bitmap, size, first + 1); + assert(next >= first); + *num = next - first; + return first; +} + static inline bool migration_bitmap_clear_dirty(RAMState *rs, RAMBlock *rb, unsigned long page) @@ -3730,19 +3765,26 @@ void colo_flush_ram_cache(void) block = QLIST_FIRST_RCU(&ram_list.blocks); while (block) { - offset = migration_bitmap_find_dirty(ram_state, block, offset); + unsigned long num = 0; + offset = colo_bitmap_find_dirty(ram_state, block, offset, &num); if (((ram_addr_t)offset) << TARGET_PAGE_BITS >= block->used_length) { offset = 0; + num = 0; block = QLIST_NEXT_RCU(block, next); } else { - migration_bitmap_clear_dirty(ram_state, block, offset); + unsigned long i = 0; + + for (i = 0; i < num; i++) { + migration_bitmap_clear_dirty(ram_state, block, offset + i); + } dst_host = block->host + (((ram_addr_t)offset) << TARGET_PAGE_BITS); src_host = block->colo_cache + (((ram_addr_t)offset) << TARGET_PAGE_BITS); - memcpy(dst_host, src_host, TARGET_PAGE_SIZE); + memcpy(dst_host, src_host, TARGET_PAGE_SIZE * num); + offset += num; } } }