From patchwork Fri May 12 11:31:40 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Perevalov X-Patchwork-Id: 9723965 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2CE6C60348 for ; Fri, 12 May 2017 11:34:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 28EF128788 for ; Fri, 12 May 2017 11:34:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1D76F287BB; Fri, 12 May 2017 11:34:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 758E428788 for ; Fri, 12 May 2017 11:34:04 +0000 (UTC) Received: from localhost ([::1]:52988 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d98pf-0003Zp-Gy for patchwork-qemu-devel@patchwork.kernel.org; Fri, 12 May 2017 07:34:03 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57370) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d98oL-0003XU-Na for qemu-devel@nongnu.org; Fri, 12 May 2017 07:32:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d98oI-0000tZ-I9 for qemu-devel@nongnu.org; Fri, 12 May 2017 07:32:41 -0400 Received: from mailout3.w1.samsung.com ([210.118.77.13]:19970) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d98oI-0000ix-89 for qemu-devel@nongnu.org; Fri, 12 May 2017 07:32:38 -0400 Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout3.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0OPU00JSG81YAD80@mailout3.w1.samsung.com> for qemu-devel@nongnu.org; Fri, 12 May 2017 12:32:22 +0100 (BST) Received: from eusmges3.samsung.com (unknown [203.254.199.242]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20170512113222eucas1p17f9771c49257bbb2921c2d070bd9a392~92IrM_vkq3190631906eucas1p1W; Fri, 12 May 2017 11:32:22 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges3.samsung.com (EUCPMTA) with SMTP id A2.C8.17464.54D95195; Fri, 12 May 2017 12:32:21 +0100 (BST) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20170512113221eucas1p2c0908a17f830fe10ce233ab2248ff0e4~92Iqkkekf0890908909eucas1p2Z; Fri, 12 May 2017 11:32:21 +0000 (GMT) X-AuditID: cbfec7f2-f797e6d000004438-55-59159d4585af Received: from eusync2.samsung.com ( [203.254.199.212]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 79.8E.17452.8DD95195; Fri, 12 May 2017 12:34:48 +0100 (BST) Received: from aperevalov-ubuntu.rnd.samsung.ru ([106.109.129.199]) by eusync2.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0OPU0053K81CCS30@eusync2.samsung.com>; Fri, 12 May 2017 12:32:21 +0100 (BST) From: Alexey Perevalov To: qemu-devel@nongnu.org Date: Fri, 12 May 2017 14:31:40 +0300 Message-id: <1494588703-9076-7-git-send-email-a.perevalov@samsung.com> X-Mailer: git-send-email 1.9.1 In-reply-to: <1494588703-9076-1-git-send-email-a.perevalov@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrGIsWRmVeSWpSXmKPExsWy7djPc7quc0UjDR6d17OYe/c8i0Xvtnvs Flfaf7JbbNn/jd3ieO8OFgdWjyfXNjN5vN93lc2jb8sqxgDmKC6blNSczLLUIn27BK6Mp5se sxa81ao417OJpYGxWamLkZNDQsBEYuu5dkYIW0ziwr31bF2MXBxCAksZJaZcnM8GkhAS+Mwo sXRhOEzDvNP7GCGKljFK7P27kB3C6WaS2HfpNEsXIwcHm4CBxL57tiANIgKSEr+7TjOD2MwC sRJvr18DGyoskCLxavscsM0sAqoS605PBrN5Bdwk1v3tZoZYJidx8thkVhCbU8Bd4sOa5WCL JQQus0m0Hr/KBLJLQkBWYtMBqHoXif6lS9ggbGGJV8e3sEPYMhKdHQeZIHrbGSW6d3ayQjgT GCXOTP8LVWUvceomyFCQS/kkJm2bzgyxgFeio00IosRD4vKvuVALHCV+/P4HDa5ZjBKv931g m8Aos4CRYRWjSGppcW56arGxXnFibnFpXrpecn7uJkZgbJ7+d/zTDsavJ6wOMQpwMCrx8EpU iUQKsSaWFVfmHmKU4GBWEuFdES8aKcSbklhZlVqUH19UmpNafIhRmoNFSZyX69S1CCGB9MSS 1OzU1ILUIpgsEwenVAPjnJIadv1963/kn5Pe6qv8yaZR51vjEbH1d6cu8j/dsfiK2uGZ5//O cTth7a/vV3rNIEtd+inb30ezhA7dT9/Bf3dxU8zvvphNC2zO525Zm9TAIRGuO6NJI+n1N/7Y 9o2qXz8mndi14sxs/7XzjCuNM3SsGKzb12cYb2fYm9SpHbUzesHC/dIaSizFGYmGWsxFxYkA Q+IyMskCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrCLMWRmVeSWpSXmKPExsVy+t/xK7o35opGGrx/q2kx9+55FovebffY La60/2S32LL/G7vF8d4dLA6sHk+ubWbyeL/vKptH35ZVjAHMUW42GamJKalFCql5yfkpmXnp tkqhIW66FkoKeYm5qbZKEbq+IUFKCmWJOaVAnpEBGnBwDnAPVtK3S3DLeLrpMWvBW62Kcz2b WBoYm5W6GDk5JARMJOad3scIYYtJXLi3nq2LkYtDSGAJo8Si+bvYIZxeJolbz0GqODjYBAwk 9t2zBWkQEZCU+N11mhnEZhaIlbg04QUbiC0skCLxavscsKEsAqoS605PBrN5Bdwk1v3tZoZY Jidx8thkVhCbU8Bd4sOa5WDjhYBq5v6LncDIu4CRYRWjSGppcW56brGhXnFibnFpXrpecn7u JkZgiG479nPzDsZLG4MPMQpwMCrx8EpUiUQKsSaWFVfmHmKU4GBWEuFdES8aKcSbklhZlVqU H19UmpNafIjRFOimicxSosn5wPjJK4k3NDE0tzQ0MrawMDcyUhLnLflwJVxIID2xJDU7NbUg tQimj4mDU6qB0fvyS+egDV2sW7hc3Z1S7pxZovxt6V+1hekKsT3ccmKbQs1nvAp+vVbijqRm 2tR4Dmv77fOSg9nC+59zTr8VKOR9/XbO/T9J79+d5rPpuqpnrrspku/VaSsVTtE7kmfuvc8K VdAMTxWJuMHcs+RPm+Taj7V6dkHKgd9vOp0+0pE6Q0ivzspLiaU4I9FQi7moOBEAYjSaVWcC AAA= X-MTR: 20000000000000000@CPGS X-CMS-MailID: 20170512113221eucas1p2c0908a17f830fe10ce233ab2248ff0e4 X-Msg-Generator: CA X-Sender-IP: 182.198.249.179 X-Local-Sender: =?UTF-8?B?QWxleGV5IFBlcmV2YWxvdhtTUlItVmlydHVhbGl6YXRpb24g?= =?UTF-8?B?TGFiG+yCvOyEseyghOyekBtTZW5pb3IgRW5naW5lZXI=?= X-Global-Sender: =?UTF-8?B?QWxleGV5IFBlcmV2YWxvdhtTUlItVmlydHVhbGl6YXRpb24g?= =?UTF-8?B?TGFiG1NhbXN1bmcgRWxlY3Ryb25pY3MbU2VuaW9yIEVuZ2luZWVy?= X-Sender-Code: =?UTF-8?B?QzEwG0NJU0hRG0MxMEdEMDFHRDAxMDE1NA==?= CMS-TYPE: 201P X-HopCount: 7 X-CMS-RootMailID: 20170512113221eucas1p2c0908a17f830fe10ce233ab2248ff0e4 X-RootMTR: 20170512113221eucas1p2c0908a17f830fe10ce233ab2248ff0e4 References: <1494588703-9076-1-git-send-email-a.perevalov@samsung.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 210.118.77.13 Subject: [Qemu-devel] [PATCH V4 6/9] migration: add postcopy vcpu blocktime context into MigrationIncomingState X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: i.maximets@samsung.com, dgilbert@redhat.com, peterx@redhat.com, a.perevalov@samsung.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch adds request to kernel space for UFFD_FEATURE_THREAD_ID, in case when this feature is provided by kernel. PostcopyBlocktimeContext is incapsulated inside postcopy-ram.c, due to it's postcopy only feature. Also it defines PostcopyBlocktimeContext's instance live time. Information from PostcopyBlocktimeContext instance will be provided much after postcopy migration end, instance of PostcopyBlocktimeContext will live till QEMU exit, but part of it (vcpu_addr, page_fault_vcpu_time) used only during calculation, will be released when postcopy ended or failed. To enable postcopy blocktime calculation on destination, need to request proper capabiltiy (Patch for documentation will at the tail of patch set). As an example following command enable that capability, assume qemu was started with -chardev socket,id=charmonitor,path=/var/lib/migrate-vm-monitor.sock option to control it [root@host]#printf "{\"execute\" : \"qmp_capabilities\"}\r\n \ {\"execute\": \"migrate-set-capabilities\" , \"arguments\": { \"capabilities\": [ { \"capability\": \"postcopy-blocktime\", \"state\": true } ] } }" | nc -U /var/lib/migrate-vm-monitor.sock Signed-off-by: Alexey Perevalov --- include/migration/migration.h | 8 +++++ migration/postcopy-ram.c | 76 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 84 insertions(+) diff --git a/include/migration/migration.h b/include/migration/migration.h index 30108d3..3f34920 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -83,6 +83,8 @@ typedef enum { POSTCOPY_INCOMING_END } PostcopyState; +struct PostcopyBlocktimeContext; + /* State for the incoming migration */ struct MigrationIncomingState { QEMUFile *from_src_file; @@ -123,6 +125,12 @@ struct MigrationIncomingState { /* See savevm.c */ LoadStateEntry_Head loadvm_handlers; + + /* + * PostcopyBlocktimeContext to keep information for postcopy + * live migration, to calculate vCPU block time + * */ + struct PostcopyBlocktimeContext *blocktime_ctx; }; MigrationIncomingState *migration_incoming_get_current(void); diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index a16cde8..fd5aadd 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -59,6 +59,73 @@ struct PostcopyDiscardState { #include #include +typedef struct PostcopyBlocktimeContext { + /* time when page fault initiated per vCPU */ + int64_t *page_fault_vcpu_time; + /* page address per vCPU */ + uint64_t *vcpu_addr; + int64_t total_blocktime; + /* blocktime per vCPU */ + int64_t *vcpu_blocktime; + /* point in time when last page fault was initiated */ + int64_t last_begin; + /* number of vCPU are suspended */ + int smp_cpus_down; + + /* + * Handler for exit event, necessary for + * releasing whole blocktime_ctx + */ + Notifier exit_notifier; + /* + * Handler for postcopy event, necessary for + * releasing unnecessary part of blocktime_ctx + */ + Notifier postcopy_notifier; +} PostcopyBlocktimeContext; + +static void destroy_blocktime_context(struct PostcopyBlocktimeContext *ctx) +{ + g_free(ctx->page_fault_vcpu_time); + g_free(ctx->vcpu_addr); + g_free(ctx->vcpu_blocktime); + g_free(ctx); +} + +static void postcopy_migration_cb(Notifier *n, void *data) +{ + PostcopyBlocktimeContext *ctx = container_of(n, PostcopyBlocktimeContext, + postcopy_notifier); + MigrationState *s = data; + if (migration_has_finished(s) || migration_has_failed(s)) { + g_free(ctx->page_fault_vcpu_time); + /* g_free is NULL robust */ + ctx->page_fault_vcpu_time = NULL; + g_free(ctx->vcpu_addr); + ctx->vcpu_addr = NULL; + } +} + +static void migration_exit_cb(Notifier *n, void *data) +{ + PostcopyBlocktimeContext *ctx = container_of(n, PostcopyBlocktimeContext, + exit_notifier); + destroy_blocktime_context(ctx); +} + +static struct PostcopyBlocktimeContext *blocktime_context_new(void) +{ + PostcopyBlocktimeContext *ctx = g_new0(PostcopyBlocktimeContext, 1); + ctx->page_fault_vcpu_time = g_new0(int64_t, smp_cpus); + ctx->vcpu_addr = g_new0(uint64_t, smp_cpus); + ctx->vcpu_blocktime = g_new0(int64_t, smp_cpus); + + ctx->exit_notifier.notify = migration_exit_cb; + ctx->postcopy_notifier.notify = postcopy_migration_cb; + qemu_add_exit_notifier(&ctx->exit_notifier); + add_migration_state_change_notifier(&ctx->postcopy_notifier); + return ctx; +} /* * Check userfault fd features, to request only supported features in @@ -133,6 +200,15 @@ static bool ufd_check_and_apply(int ufd, MigrationIncomingState *mis) return false; } +#ifdef UFFD_FEATURE_THREAD_ID + if (migrate_postcopy_blocktime() && mis && + UFFD_FEATURE_THREAD_ID & supported_features) { + /* kernel supports that feature */ + mis->blocktime_ctx = blocktime_context_new(); + asked_features |= UFFD_FEATURE_THREAD_ID; + } +#endif + /* * request features, even if asked_features is 0, due to * kernel expects UFFD_API before UFFDIO_REGISTER, per