From patchwork Sun Aug 21 20:58:50 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Md Haris Iqbal X-Patchwork-Id: 9292709 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8AB95607FF for ; Sun, 21 Aug 2016 21:00:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 78B1F283E4 for ; Sun, 21 Aug 2016 21:00:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6971D2889B; Sun, 21 Aug 2016 21:00:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CD343283E4 for ; Sun, 21 Aug 2016 21:00:00 +0000 (UTC) Received: from localhost ([::1]:38168 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bbZqa-0007ci-1s for patchwork-qemu-devel@patchwork.kernel.org; Sun, 21 Aug 2016 17:00:00 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60508) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bbZpx-0007TD-UM for qemu-devel@nongnu.org; Sun, 21 Aug 2016 16:59:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bbZpv-0005zC-Kd for qemu-devel@nongnu.org; Sun, 21 Aug 2016 16:59:20 -0400 Received: from mail-pf0-x243.google.com ([2607:f8b0:400e:c00::243]:35984) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bbZpv-0005yw-Ah for qemu-devel@nongnu.org; Sun, 21 Aug 2016 16:59:19 -0400 Received: by mail-pf0-x243.google.com with SMTP id y134so5013343pfg.3 for ; Sun, 21 Aug 2016 13:59:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=QLFp2wntM7WnEZEtQeCk6qYDLeCGu1wIr1ICBzgn/yc=; b=UnHFb7coNvLCKZwpTpQfqv+fT11fqMcBDdYg8SjCSosvTaIRrn8QbH5GomYeVJPwia nWMJWnfZ4AI5eYQRqDftNNiPj8k4eoIzbwjt2QWstv4WFnDm5ygNZNG1/r9A8hhpXAJZ 8zHtbc4xUY3QQM/LK44m7dvWEayZl7swgA2dCsfqA2l6rmsq+5p8RrCPgD1kfThmqH92 L7Gtz5Z49xuNKm6PuawHHFRMxPBVAenMxhMZwn7pgYbWbrAeBtORWLt5StDzh9UOmQDS o79U+PF0BZH8MoBYwWOPzXvLkMIndTKPvOh7tOiitpV6OMC0iJWGH8xBkiN0MAXqdihP 5szg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=QLFp2wntM7WnEZEtQeCk6qYDLeCGu1wIr1ICBzgn/yc=; b=GRvy4egdZt9s+qrY9X2FPzhDB7lWexzrWgehF6DhlF3PpOXLDG6WNg/YdXIow0qIxq eAjegh3ezfwVmc8mhVUNUHxCn5WdwaP1GNutN0PTYeVzjz3uV82m8UdPyizuM1GBO9Z6 HPjHip0Fu1/bez2gzWmgR20Sje66xH2gNpNsxjMWeTBvnMcVObm7QbYrkxtH1HVU/9sT KOFFtXNcXg3Ns+b8sSTfrritVi59vE7L5pbqgpjiIOC8DHp7jZyoUCbvQt2nnNpO8PD3 yGk2ocy/IZOr7LeIlTkKunBEEwbdvAybsv244idJN33WO3cpQ9GuV3n87tGNynrMIXdW wHkw== X-Gm-Message-State: AEkoouvoMloqrbI0bFs9qHpOgrS1sPYs8dglgpEVM1ffz1EtclKIzlyWRl99TAlYMGgtqg== X-Received: by 10.98.135.13 with SMTP id i13mr36494581pfe.44.1471813158333; Sun, 21 Aug 2016 13:59:18 -0700 (PDT) Received: from deepak.domain.name ([103.252.25.179]) by smtp.googlemail.com with ESMTPSA id ra13sm27025890pac.29.2016.08.21.13.59.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 21 Aug 2016 13:59:17 -0700 (PDT) From: Md Haris Iqbal To: qemu-devel@nongnu.org Date: Mon, 22 Aug 2016 02:28:50 +0530 Message-Id: <1471813132-13836-5-git-send-email-haris.phnx@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1471813132-13836-1-git-send-email-haris.phnx@gmail.com> References: <1471813132-13836-1-git-send-email-haris.phnx@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2607:f8b0:400e:c00::243 Subject: [Qemu-devel] [PATCH 4/6] Migration: New bitmap for postcopy migration failure X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Md Haris Iqbal , dgilbert@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Md Haris Iqbal --- include/migration/migration.h | 4 +++ migration/migration.c | 4 +++ migration/postcopy-ram.c | 3 ++ migration/ram.c | 73 ++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 83 insertions(+), 1 deletion(-) diff --git a/include/migration/migration.h b/include/migration/migration.h index 74d456e..4e4c0c8 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -358,6 +358,10 @@ int ram_save_queue_pages(MigrationState *ms, const char *rbname, int qemu_migrate_postcopy_outgoing_recovery(MigrationState *ms); int qemu_migrate_postcopy_incoming_recovery(QEMUFile **f,MigrationIncomingState* mis); +void migrate_incoming_ram_bitmap_init(void); +void migrate_incoming_ram_bitmap_update(RAMBlock *rb, ram_addr_t addr); +void migrate_incoming_ram_bitmap_free(void); + PostcopyState postcopy_state_get(void); /* Set the state and return the old state */ PostcopyState postcopy_state_set(PostcopyState new_state); diff --git a/migration/migration.c b/migration/migration.c index 166f4f7..7cd3344 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -381,6 +381,10 @@ static void process_incoming_migration_co(void *opaque) postcopy_state_set(POSTCOPY_INCOMING_NONE); migrate_set_state(&mis->state, MIGRATION_STATUS_NONE, MIGRATION_STATUS_ACTIVE); + + /* Initializing the bitmap for destination side */ + migrate_incoming_ram_bitmap_init(); + ret = qemu_loadvm_state(f); ps = postcopy_state_get(); diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index d19c13a..a8bb311 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -317,6 +317,9 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingState *mis) postcopy_state_set(POSTCOPY_INCOMING_END); migrate_send_rp_shut(mis, qemu_file_get_error(mis->from_src_file) != 0); + /* Free the bitmap used to keep track of incoming pages */ + migrate_incoming_ram_bitmap_free(); + if (mis->postcopy_tmp_page) { munmap(mis->postcopy_tmp_page, getpagesize()); mis->postcopy_tmp_page = NULL; diff --git a/migration/ram.c b/migration/ram.c index a3d70c4..ea1382b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -250,6 +250,13 @@ static struct BitmapRcu { * of the postcopy phase */ unsigned long *unsentmap; + /* + * A new bitmap for postcopy network failure recovery. + * It keeps track of the pages recieved. + * In the end, it would be used to request pages that were + * lost due to network failure. + */ + unsigned long *not_received; } *migration_bitmap_rcu; struct CompressParam { @@ -2340,6 +2347,7 @@ static int ram_load_postcopy(QEMUFile *f) void *page_buffer = NULL; void *place_source = NULL; uint8_t ch; + RAMBlock* block = NULL; addr = qemu_get_be64(f); flags = addr & ~TARGET_PAGE_MASK; @@ -2348,7 +2356,7 @@ static int ram_load_postcopy(QEMUFile *f) trace_ram_load_postcopy_loop((uint64_t)addr, flags); place_needed = false; if (flags & (RAM_SAVE_FLAG_COMPRESS | RAM_SAVE_FLAG_PAGE)) { - RAMBlock *block = ram_block_from_stream(f, flags); + block = ram_block_from_stream(f, flags); host = host_from_ram_block_offset(block, addr); if (!host) { @@ -2436,6 +2444,15 @@ static int ram_load_postcopy(QEMUFile *f) if (!ret) { ret = qemu_file_get_error(f); } + if (block != NULL) { + /* + * TODO + * We need to delay updating the bits until host page is + * recieved and the place is done, or tidy up the bitmap later + * accordingly (whether whole host page was recieved or not) + */ + migrate_incoming_ram_bitmap_update(block, addr); + } } return ret; @@ -2483,6 +2500,16 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id) RAMBlock *block = ram_block_from_stream(f, flags); host = host_from_ram_block_offset(block, addr); + + migrate_incoming_ram_bitmap_update(block, addr); + /* + * TODO + * 1) Do we need a bitmap_update call later in the while loop also? + * 2) We need to delay updating the bits until host page is + * recieved and the place is done, or tidy up the bitmap later + * accordingly (whether whole host page was recieved or not) + */ + if (!host) { error_report("Illegal RAM offset " RAM_ADDR_FMT, addr); ret = -EINVAL; @@ -2578,6 +2605,50 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id) return ret; } +void migrate_incoming_ram_bitmap_init(void) +{ + int64_t ram_bitmap_pages; /* Size of bitmap in pages, including gaps */ + + /* + * A new bitmap for postcopy network failure recovery. + * It keeps track of the pages recieved. + * In the end, it would be used to request pages that were + * lost due to network failure. + */ + + ram_bitmap_pages = last_ram_offset() >> TARGET_PAGE_BITS; + migration_bitmap_rcu = g_new0(struct BitmapRcu, 1); + migration_bitmap_rcu->not_received = bitmap_new(ram_bitmap_pages); + bitmap_set(migration_bitmap_rcu->not_received, 0, ram_bitmap_pages); +} + +void migrate_incoming_ram_bitmap_update(RAMBlock *rb, ram_addr_t addr) +{ + unsigned long base = rb->offset >> TARGET_PAGE_BITS; + unsigned long nr = base + (addr >> TARGET_PAGE_BITS); + unsigned long *bitmap; + + bitmap = atomic_rcu_read(&migration_bitmap_rcu)->not_received; + clear_bit(nr, bitmap); + + static int count = 0; + count++; + if(count == 1000) { + count = 0; + ram_debug_dump_bitmap(bitmap, true); + } +} + +void migrate_incoming_ram_bitmap_free(void) +{ + struct BitmapRcu *bitmap = migration_bitmap_rcu; + atomic_rcu_set(&migration_bitmap_rcu, NULL); + if (bitmap) { + memory_global_dirty_log_stop(); + call_rcu(bitmap, migration_bitmap_free, rcu); + } +} + static SaveVMHandlers savevm_ram_handlers = { .save_live_setup = ram_save_setup, .save_live_iterate = ram_save_iterate,