From patchwork Wed Jun 7 09:46:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Perevalov X-Patchwork-Id: 9771149 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 159BC6034B for ; Wed, 7 Jun 2017 09:50:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DFA6C28459 for ; Wed, 7 Jun 2017 09:50:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D42962849E; Wed, 7 Jun 2017 09:50:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1D33E284EE for ; Wed, 7 Jun 2017 09:50:26 +0000 (UTC) Received: from localhost ([::1]:42316 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dIXbd-0002G3-8y for patchwork-qemu-devel@patchwork.kernel.org; Wed, 07 Jun 2017 05:50:25 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58037) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dIXYt-0001Aq-DU for qemu-devel@nongnu.org; Wed, 07 Jun 2017 05:47:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dIXYp-0000F8-Qn for qemu-devel@nongnu.org; Wed, 07 Jun 2017 05:47:35 -0400 Received: from mailout2.w1.samsung.com ([210.118.77.12]:15626) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dIXYp-0000EQ-LE for qemu-devel@nongnu.org; Wed, 07 Jun 2017 05:47:31 -0400 Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0OR6008FQ8J61630@mailout2.w1.samsung.com> for qemu-devel@nongnu.org; Wed, 07 Jun 2017 10:47:30 +0100 (BST) Received: from eusmges1.samsung.com (unknown [203.254.199.239]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20170607094729eucas1p2b162ae008609f30230018a20c3a5d256~FzeiSJ5iJ0825708257eucas1p2K; Wed, 7 Jun 2017 09:47:29 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges1.samsung.com (EUCPMTA) with SMTP id 9E.CE.14140.4BBC7395; Wed, 7 Jun 2017 10:47:32 +0100 (BST) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20170607094729eucas1p119b3d77f7d869eb06c16c9e91215e8cd~FzehsVLPS2189821898eucas1p1b; Wed, 7 Jun 2017 09:47:29 +0000 (GMT) X-AuditID: cbfec7ef-f796a6d00000373c-2f-5937cbb4b9ab Received: from eusync1.samsung.com ( [203.254.199.211]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id CF.22.20206.1BBC7395; Wed, 7 Jun 2017 10:47:29 +0100 (BST) Received: from aperevalov-ubuntu.rnd.samsung.ru ([106.109.129.199]) by eusync1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0OR600I3B8ILMHB0@eusync1.samsung.com>; Wed, 07 Jun 2017 10:47:29 +0100 (BST) From: Alexey Perevalov To: qemu-devel@nongnu.org Date: Wed, 07 Jun 2017 12:46:33 +0300 Message-id: <1496828798-27548-7-git-send-email-a.perevalov@samsung.com> X-Mailer: git-send-email 1.9.1 In-reply-to: <1496828798-27548-1-git-send-email-a.perevalov@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrGIsWRmVeSWpSXmKPExsWy7djP87pbTptHGsxeJWwx9+55FovebffY La60/2S32LL/G7vF8d4dLA6sHk+ubWbyeL/vKptH35ZVjAHMUVw2Kak5mWWpRfp2CVwZ/5te sxQs0ag4+O0kWwPjXIUuRk4OCQETidm/TrNA2GISF+6tZ+ti5OIQEljGKDFv6QdmkISQwGdG iafbFWEa3mxoZIErWrVnLjOE080ksXzBWSCHg4NNwEBi3z1bkAYRAUmJ312nwQYxCxRIfGxd CbZNWCBW4tuTqYwgNouAqsTqPz1sIDavgLvEplVTmSGWyUmcPDaZFcTmFPCQ+DH1NtguCYHb bBKb9x5jAdklISArsekAVL2LxLw1txghbGGJV8e3sEPYMhKXJ3ezQPS2M0p07+xkhXAmMEqc mf4Xqspe4tTNq0wQl/JJTNo2nRliAa9ER5sQRImHxMumiVDljhIbj8xlgnh+NqNE//1HTBMY ZRYwMqxiFEktLc5NTy021CtOzC0uzUvXS87P3cQIjM3T/46/38H4tDnkEKMAB6MSD2/GHrNI IdbEsuLK3EOMEhzMSiK8E4+aRwrxpiRWVqUW5ccXleakFh9ilOZgURLn5T11LUJIID2xJDU7 NbUgtQgmy8TBKdXA2H5xw9vIp9+9Q5VOhidOTN3xItHz+6O4c7V6E1bPFeafWWUa9XZdXe2E txbp0dUy7wrClXo21C0K2lL5VUPlQ4z93BmL57LuOv6ktvbLrHmiyTff7b3A31D+03qtdotD jJvna9aEF58KAxr2Gb5xveY87cz5DrtE0+LPO0z2Rs3TMDpvrnY6XYmlOCPRUIu5qDgRAH9V V3bJAgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrKLMWRmVeSWpSXmKPExsVy+t/xy7obT5tHGsw6wWwx9+55FovebffY La60/2S32LL/G7vF8d4dLA6sHk+ubWbyeL/vKptH35ZVjAHMUW42GamJKalFCql5yfkpmXnp tkqhIW66FkoKeYm5qbZKEbq+IUFKCmWJOaVAnpEBGnBwDnAPVtK3S3DL+N/0mqVgiUbFwW8n 2RoY5yp0MXJySAiYSLzZ0MgCYYtJXLi3nq2LkYtDSGAJo8SRlQ2sEE4vk8Tug9+Yuhg5ONgE DCT23bMFaRARkJT43XWaGcRmFiiQODd5CtggYYFYiW9PpjKC2CwCqhKr//Swgdi8Au4Sm1ZN ZYZYJidx8thkVhCbU8BD4sfU22BxIaCa23fOME1g5F3AyLCKUSS1tDg3PbfYSK84Mbe4NC9d Lzk/dxMjMEi3Hfu5ZQdj17vgQ4wCHIxKPLwZe8wihVgTy4orcw8xSnAwK4nwTjxqHinEm5JY WZValB9fVJqTWnyI0RToqInMUqLJ+cAIyiuJNzQxNLc0NDK2sDA3MlIS55364Uq4kEB6Yklq dmpqQWoRTB8TB6dUA+NSzw07Z17TNLu4rV529pJpwgq+r83u7xCfpa+5tMNZr2pBbViY9u4m Pa2Mc7PeMr3Z2zGjaMOPC/fj75nxcTr9+M2/R/yuVC1zrM6po5mZ9y3VI5c9DVxxIaKZb9fb tHutqeKnkwqMsvb+nvptUeDZ5zxyMix1BXfPLLz1kGmH2gVth8dV/buVWIozEg21mIuKEwHm cW9maAIAAA== X-MTR: 20000000000000000@CPGS X-CMS-MailID: 20170607094729eucas1p119b3d77f7d869eb06c16c9e91215e8cd X-Msg-Generator: CA X-Sender-IP: 182.198.249.180 X-Local-Sender: =?UTF-8?B?QWxleGV5IFBlcmV2YWxvdhtTUlItVmlydHVhbGl6YXRpb24g?= =?UTF-8?B?TGFiG+yCvOyEseyghOyekBtTZW5pb3IgRW5naW5lZXI=?= X-Global-Sender: =?UTF-8?B?QWxleGV5IFBlcmV2YWxvdhtTUlItVmlydHVhbGl6YXRpb24g?= =?UTF-8?B?TGFiG1NhbXN1bmcgRWxlY3Ryb25pY3MbU2VuaW9yIEVuZ2luZWVy?= X-Sender-Code: =?UTF-8?B?QzEwG0NJU0hRG0MxMEdEMDFHRDAxMDE1NA==?= CMS-TYPE: 201P X-HopCount: 7 X-CMS-RootMailID: 20170607094729eucas1p119b3d77f7d869eb06c16c9e91215e8cd X-RootMTR: 20170607094729eucas1p119b3d77f7d869eb06c16c9e91215e8cd References: <1496828798-27548-1-git-send-email-a.perevalov@samsung.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 210.118.77.12 Subject: [Qemu-devel] [PATCH v8 06/11] migration: add postcopy blocktime ctx into MigrationIncomingState X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: i.maximets@samsung.com, Alexey Perevalov , peterx@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch adds request to kernel space for UFFD_FEATURE_THREAD_ID, in case when this feature is provided by kernel. PostcopyBlocktimeContext is incapsulated inside postcopy-ram.c, due to it's postcopy only feature. Also it defines PostcopyBlocktimeContext's instance live time. Information from PostcopyBlocktimeContext instance will be provided much after postcopy migration end, instance of PostcopyBlocktimeContext will live till QEMU exit, but part of it (vcpu_addr, page_fault_vcpu_time) used only during calculation, will be released when postcopy ended or failed. To enable postcopy blocktime calculation on destination, need to request proper capabiltiy (Patch for documentation will be at the tail of the patch set). As an example following command enable that capability, assume QEMU was started with -chardev socket,id=charmonitor,path=/var/lib/migrate-vm-monitor.sock option to control it [root@host]#printf "{\"execute\" : \"qmp_capabilities\"}\r\n \ {\"execute\": \"migrate-set-capabilities\" , \"arguments\": { \"capabilities\": [ { \"capability\": \"postcopy-blocktime\", \"state\": true } ] } }" | nc -U /var/lib/migrate-vm-monitor.sock Or just with HMP (qemu) migrate_set_capability postcopy-blocktime on Signed-off-by: Alexey Perevalov --- include/migration/migration.h | 8 ++++++ migration/postcopy-ram.c | 65 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 73 insertions(+) diff --git a/include/migration/migration.h b/include/migration/migration.h index 2e61df5..766e802 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -49,6 +49,8 @@ enum mig_rp_message_type { MIG_RP_MSG_MAX }; +struct PostcopyBlocktimeContext; + /* State for the incoming migration */ struct MigrationIncomingState { QEMUFile *from_src_file; @@ -86,6 +88,12 @@ struct MigrationIncomingState { /* The coroutine we should enter (back) after failover */ Coroutine *migration_incoming_co; QemuSemaphore colo_incoming_sem; + + /* + * PostcopyBlocktimeContext to keep information for postcopy + * live migration, to calculate vCPU block time + * */ + struct PostcopyBlocktimeContext *blocktime_ctx; }; MigrationIncomingState *migration_incoming_get_current(void); diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index cbe8f9f..ade7f1c 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -63,6 +63,58 @@ struct PostcopyDiscardState { #include #include +typedef struct PostcopyBlocktimeContext { + /* time when page fault initiated per vCPU */ + int64_t *page_fault_vcpu_time; + /* page address per vCPU */ + uint64_t *vcpu_addr; + int64_t total_blocktime; + /* blocktime per vCPU */ + int64_t *vcpu_blocktime; + /* point in time when last page fault was initiated */ + int64_t last_begin; + /* number of vCPU are suspended */ + int smp_cpus_down; + + /* + * Handler for exit event, necessary for + * releasing whole blocktime_ctx + */ + Notifier exit_notifier; + /* + * Handler for postcopy event, necessary for + * releasing unnecessary part of blocktime_ctx + */ + Notifier postcopy_notifier; +} PostcopyBlocktimeContext; + +static void destroy_blocktime_context(struct PostcopyBlocktimeContext *ctx) +{ + g_free(ctx->page_fault_vcpu_time); + g_free(ctx->vcpu_addr); + g_free(ctx->vcpu_blocktime); + g_free(ctx); +} + +static void migration_exit_cb(Notifier *n, void *data) +{ + PostcopyBlocktimeContext *ctx = container_of(n, PostcopyBlocktimeContext, + exit_notifier); + destroy_blocktime_context(ctx); +} + +static struct PostcopyBlocktimeContext *blocktime_context_new(void) +{ + PostcopyBlocktimeContext *ctx = g_new0(PostcopyBlocktimeContext, 1); + ctx->page_fault_vcpu_time = g_new0(int64_t, smp_cpus); + ctx->vcpu_addr = g_new0(uint64_t, smp_cpus); + ctx->vcpu_blocktime = g_new0(int64_t, smp_cpus); + + ctx->exit_notifier.notify = migration_exit_cb; + qemu_add_exit_notifier(&ctx->exit_notifier); + add_migration_state_change_notifier(&ctx->postcopy_notifier); + return ctx; +} /** * receive_ufd_features: check userfault fd features, to request only supported @@ -155,6 +207,19 @@ static bool ufd_check_and_apply(int ufd, MigrationIncomingState *mis) } } +#ifdef UFFD_FEATURE_THREAD_ID + if (migrate_postcopy_blocktime() && mis && + UFFD_FEATURE_THREAD_ID & supported_features) { + /* kernel supports that feature */ + /* don't create blocktime_context if it exists */ + if (!mis->blocktime_ctx) { + mis->blocktime_ctx = blocktime_context_new(); + } + + asked_features |= UFFD_FEATURE_THREAD_ID; + } +#endif + /* * request features, even if asked_features is 0, due to * kernel expects UFFD_API before UFFDIO_REGISTER, per