From patchwork Mon Sep 3 04:38:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 10585413 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 43522174C for ; Mon, 3 Sep 2018 04:46:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 29B24296D0 for ; Mon, 3 Sep 2018 04:46:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1AF20296D4; Mon, 3 Sep 2018 04:46:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A4A05296D0 for ; Mon, 3 Sep 2018 04:46:02 +0000 (UTC) Received: from localhost ([::1]:43389 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fwgkT-0006B7-Ly for patchwork-qemu-devel@patchwork.kernel.org; Mon, 03 Sep 2018 00:46:01 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56256) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fwgfb-0008Fo-5g for qemu-devel@nongnu.org; Mon, 03 Sep 2018 00:41:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fwgfa-0004n8-C4 for qemu-devel@nongnu.org; Mon, 03 Sep 2018 00:40:59 -0400 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]:35272) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fwgfa-0004mq-4Z for qemu-devel@nongnu.org; Mon, 03 Sep 2018 00:40:58 -0400 Received: by mail-pl1-x62d.google.com with SMTP id d9-v6so8052145plr.2 for ; Sun, 02 Sep 2018 21:40:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Exd1IDdhmKszy63Ebko1M5F8PB1LeR9RlGujcgIUx5o=; b=rVaCUcdMV7iEY0a5qPvuDogt5RiDAJef1ulzJBcpNb0Qdv5usXs7RXyw0R6DyghcgD tyGNIb91d9r6mLL7PWoZBCrz+J/JP97ByvPa2p4lq4TFaCuBCs3CcL8h5TN8eLa+6hf6 X18QBEsK0SvBe+K6yyyMc6tLuHfDs/jxYxlgV0GA8OkRohE8NJgKCWLwXE6bZds/F4ne pCZoFxIlWuse09ysvj75pURRB3to3OlSM0WoSEGSQroldEvm6eVlwTSpmcw+VHFxWJUj JZGEFDC6d5us73sUdD6WCWxsHTBrwhRL5NyLRqfuAnenxdZV6PvgCWJ7aNJRs6VKl2FV Av8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Exd1IDdhmKszy63Ebko1M5F8PB1LeR9RlGujcgIUx5o=; b=bGaB2I0x7YvYs+y2LLEWIvwmODHQTijmy/M4I+2JmXRGojPabBdci/3A68yVxfgVNB 9Ox5jM15G6EAo48cT/qxtkFGD/HiyLJps6WOiKeZS5V9zPz7Lu2V7RHOIWPCt+dMO7pm RNfi/ETtPChGY+a9+vblMzboKv+aNYEmVEvGcZ+RX73J4n4U/J/KReZ4ZywD0Xao40dp DwAXGuNf4ouy/avADVH+Sc24GArp0RYas4/wu5A0Bave1NwpU/Nycatxwf49TcPFWSsI SDs5tIKZf6FzU1ImIBjkOuxEQ5GJyEeYACECrB5FBiuFTcIxF7iSZ/MGdtqYyGFALxi+ sAUg== X-Gm-Message-State: APzg51AOjmbB0sPCGxRupuGEErn6VoF6VipDLIAvRtc5qclaQOQKTc5T wozLyiS3cH1qb6PkNub41+vsMdgT X-Google-Smtp-Source: ANB0VdYIXFiLzuCueQyR7NXKs4elxM4rku24JGuA7/CwwzF4DXK9a7eqoubo3nu0/0obXGcb6Uf3KA== X-Received: by 2002:a17:902:bf46:: with SMTP id u6-v6mr26608943pls.85.1535949657035; Sun, 02 Sep 2018 21:40:57 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id g6-v6sm32738308pfb.11.2018.09.02.21.40.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 02 Sep 2018 21:40:56 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Mon, 3 Sep 2018 12:38:49 +0800 Message-Id: <20180903043900.28592-9-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180903043900.28592-1-zhangckid@gmail.com> References: <20180903043900.28592-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::62d Subject: [Qemu-devel] [PATCH V12 08/19] ram/COLO: Record the dirty pages that SVM received X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP We record the address of the dirty pages that received, it will help flushing pages that cached into SVM. Here, it is a trick, we record dirty pages by re-using migration dirty bitmap. In the later patch, we will start the dirty log for SVM, just like migration, in this way, we can record both the dirty pages caused by PVM and SVM, we only flush those dirty pages from RAM cache while do checkpoint. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 43 ++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 40 insertions(+), 3 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index a63315533e..df8202216f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3409,6 +3409,15 @@ static inline void *colo_cache_from_block_offset(RAMBlock *block, __func__, block->idstr); return NULL; } + + /* + * During colo checkpoint, we need bitmap of these migrated pages. + * It help us to decide which pages in ram cache should be flushed + * into VM's RAM later. + */ + if (!test_and_set_bit(offset >> TARGET_PAGE_BITS, block->bmap)) { + ram_state->migration_dirty_pages++; + } return block->colo_cache + offset; } @@ -3626,7 +3635,7 @@ int colo_init_ram_cache(void) RAMBlock *block; rcu_read_lock(); - QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + RAMBLOCK_FOREACH_MIGRATABLE(block) { block->colo_cache = qemu_anon_ram_alloc(block->used_length, NULL, false); @@ -3639,10 +3648,29 @@ int colo_init_ram_cache(void) memcpy(block->colo_cache, block->host, block->used_length); } rcu_read_unlock(); + /* + * Record the dirty pages that sent by PVM, we use this dirty bitmap together + * with to decide which page in cache should be flushed into SVM's RAM. Here + * we use the same name 'ram_bitmap' as for migration. + */ + if (ram_bytes_total()) { + RAMBlock *block; + + RAMBLOCK_FOREACH_MIGRATABLE(block) { + unsigned long pages = block->max_length >> TARGET_PAGE_BITS; + + block->bmap = bitmap_new(pages); + bitmap_set(block->bmap, 0, pages); + } + } + ram_state = g_new0(RAMState, 1); + ram_state->migration_dirty_pages = 0; + return 0; out_locked: - QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + + RAMBLOCK_FOREACH_MIGRATABLE(block) { if (block->colo_cache) { qemu_anon_ram_free(block->colo_cache, block->used_length); block->colo_cache = NULL; @@ -3658,14 +3686,23 @@ void colo_release_ram_cache(void) { RAMBlock *block; + RAMBLOCK_FOREACH_MIGRATABLE(block) { + g_free(block->bmap); + block->bmap = NULL; + } + rcu_read_lock(); - QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + + RAMBLOCK_FOREACH_MIGRATABLE(block) { if (block->colo_cache) { qemu_anon_ram_free(block->colo_cache, block->used_length); block->colo_cache = NULL; } } + rcu_read_unlock(); + g_free(ram_state); + ram_state = NULL; } /**