From patchwork Wed Jun 7 07:35:26 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Perevalov X-Patchwork-Id: 9770741 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C469360234 for ; Wed, 7 Jun 2017 07:37:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF17D22376 for ; Wed, 7 Jun 2017 07:37:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A3FA028521; Wed, 7 Jun 2017 07:37:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1D1BE2848D for ; Wed, 7 Jun 2017 07:37:17 +0000 (UTC) Received: from localhost ([::1]:41651 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dIVWm-00049K-5X for patchwork-qemu-devel@patchwork.kernel.org; Wed, 07 Jun 2017 03:37:16 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54576) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dIVVh-00047u-R7 for qemu-devel@nongnu.org; Wed, 07 Jun 2017 03:36:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dIVVe-0000st-Ls for qemu-devel@nongnu.org; Wed, 07 Jun 2017 03:36:09 -0400 Received: from mailout3.w1.samsung.com ([210.118.77.13]:52390) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dIVVe-0000rq-Ec for qemu-devel@nongnu.org; Wed, 07 Jun 2017 03:36:06 -0400 Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout3.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0OR600JBF2G57080@mailout3.w1.samsung.com> for qemu-devel@nongnu.org; Wed, 07 Jun 2017 08:36:05 +0100 (BST) Received: from eusmges1.samsung.com (unknown [203.254.199.239]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20170607073605eucas1p158dc03b2dba28cbb4b04eb138af38227~Fxry8v-8C2015320153eucas1p1P; Wed, 7 Jun 2017 07:36:05 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges1.samsung.com (EUCPMTA) with SMTP id 06.61.14140.7ECA7395; Wed, 7 Jun 2017 08:36:07 +0100 (BST) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20170607073604eucas1p225626eb74bee996f8d8c3407b2ce8403~FxryVHkHz0637006370eucas1p2Q; Wed, 7 Jun 2017 07:36:04 +0000 (GMT) X-AuditID: cbfec7ef-f796a6d00000373c-dc-5937ace7b1b1 Received: from eusync4.samsung.com ( [203.254.199.214]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 1B.A5.17452.4ECA7395; Wed, 7 Jun 2017 08:36:04 +0100 (BST) Received: from aperevalov-ubuntu.rnd.samsung.ru ([106.109.129.199]) by eusync4.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0OR600HQN2FH4N90@eusync4.samsung.com>; Wed, 07 Jun 2017 08:36:04 +0100 (BST) From: Alexey Perevalov To: qemu-devel@nongnu.org Date: Wed, 07 Jun 2017 10:35:26 +0300 Message-id: <1496820931-27416-7-git-send-email-a.perevalov@samsung.com> X-Mailer: git-send-email 1.9.1 In-reply-to: <1496820931-27416-1-git-send-email-a.perevalov@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrGIsWRmVeSWpSXmKPExsWy7djPc7rP15hHGjScl7GYe/c8i0Xvtnvs Flfaf7JbbNn/jd3ieO8OFgdWjyfXNjN5vN93lc2jb8sqxgDmKC6blNSczLLUIn27BK6M/02v WQqWaFQc/HaSrYFxrkIXIweHhICJxMZ+2y5GTiBTTOLCvfVsXYxcHEICyxglVrxczwThfGaU 2P3qBSNElYnEsVO3mOGqrvyezQ7hdDNJrF64mwVkLJuAgcS+e2BjRQQkJX53nWYGsZkFCiQ+ tq5kAbGFBeIlPr64yAZiswioSszv+sMEYvMKuEtsefUKapmcxMljk1lBbE4BD4l9H9aCXSQh cJ1N4tK55WwQL8hKbDrADFHvIrFi2zJ2CFtY4tXxLVC2jERnx0Go3nZGie6dnawQzgRGiTPT /0JV2UucunmVCeJSPolJ26YzQyzglehoE4Io8ZCYfHM1E4TtKPFl7U9GiOdnM0qs/7SddQKj zAJGhlWMIqmlxbnpqcWGesWJucWleel6yfm5mxiBsXn63/H3OxifNoccYhTgYFTi4c3YYxYp xJpYVlyZe4hRgoNZSYT30lLzSCHelMTKqtSi/Pii0pzU4kOM0hwsSuK8vKeuRQgJpCeWpGan phakFsFkmTg4pRoYhUTMn05Rdbs90/Lq1f/LPNm5rao1TDjSGVevn5P9QfHkldoFb/c4sIo8 zVxT1c3fdTPn5oyv05VaWi0uFCbeneju7OVt/F5nffa54wFbZoTan7XiCPpp1j3v6+zLgSaf 3b13Kj/qvdj1qS4ySmLONbno+IRpLOkSVjeS1HembuN/Yy2ktdVaiaU4I9FQi7moOBEA+5iG QskCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrKLMWRmVeSWpSXmKPExsVy+t/xa7pP1phHGnx8LWEx9+55FovebffY La60/2S32LL/G7vF8d4dLA6sHk+ubWbyeL/vKptH35ZVjAHMUW42GamJKalFCql5yfkpmXnp tkqhIW66FkoKeYm5qbZKEbq+IUFKCmWJOaVAnpEBGnBwDnAPVtK3S3DL+N/0mqVgiUbFwW8n 2RoY5yp0MXJySAiYSBw7dYsZwhaTuHBvPVsXIxeHkMASRom5q9ewQji9TBI7f+8Gcjg42AQM JPbdswVpEBGQlPjddRqsmVmgQOLc5CksILawQLzExxcX2UBsFgFVifldf5hAbF4Bd4ktr14x QiyTkzh5bDIriM0p4CGx78NasBohoJrdp2ewTWDkXcDIsIpRJLW0ODc9t9hQrzgxt7g0L10v OT93EyMwSLcd+7l5B+OljcGHGAU4GJV4eAV2mUUKsSaWFVfmHmKU4GBWEuG9tNQ8Uog3JbGy KrUoP76oNCe1+BCjKdBRE5mlRJPzgRGUVxJvaGJobmloZGxhYW5kpCTOW/LhSriQQHpiSWp2 ampBahFMHxMHp1QDo5yI+/t0oNkOvzWjhIU6Vhz2iP6yMTjXKOlp4tObfzZ9a3bWqGzwMz4r 2GG4Ieev2ZZ99/e4KaYLuncd7ZuRZrBFvMnh7I9LX/lWlWtcWHVrtk7AkuSlyxuecJy69iZz 348VB1Ze+xPvJ8Gx8OsK4VyFyk4lTvUf188zdG6Vay9Y3TDbf8NrJZbijERDLeai4kQA3F7K 4mgCAAA= X-MTR: 20000000000000000@CPGS X-CMS-MailID: 20170607073604eucas1p225626eb74bee996f8d8c3407b2ce8403 X-Msg-Generator: CA X-Sender-IP: 182.198.249.179 X-Local-Sender: =?UTF-8?B?QWxleGV5IFBlcmV2YWxvdhtTUlItVmlydHVhbGl6YXRpb24g?= =?UTF-8?B?TGFiG+yCvOyEseyghOyekBtTZW5pb3IgRW5naW5lZXI=?= X-Global-Sender: =?UTF-8?B?QWxleGV5IFBlcmV2YWxvdhtTUlItVmlydHVhbGl6YXRpb24g?= =?UTF-8?B?TGFiG1NhbXN1bmcgRWxlY3Ryb25pY3MbU2VuaW9yIEVuZ2luZWVy?= X-Sender-Code: =?UTF-8?B?QzEwG0NJU0hRG0MxMEdEMDFHRDAxMDE1NA==?= CMS-TYPE: 201P X-HopCount: 7 X-CMS-RootMailID: 20170607073604eucas1p225626eb74bee996f8d8c3407b2ce8403 X-RootMTR: 20170607073604eucas1p225626eb74bee996f8d8c3407b2ce8403 References: <1496820931-27416-1-git-send-email-a.perevalov@samsung.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 210.118.77.13 Subject: [Qemu-devel] [[PATCH V7] 06/11] migration: add postcopy blocktime ctx into MigrationIncomingState X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: i.maximets@samsung.com, Alexey Perevalov , peterx@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch adds request to kernel space for UFFD_FEATURE_THREAD_ID, in case when this feature is provided by kernel. PostcopyBlocktimeContext is incapsulated inside postcopy-ram.c, due to it's postcopy only feature. Also it defines PostcopyBlocktimeContext's instance live time. Information from PostcopyBlocktimeContext instance will be provided much after postcopy migration end, instance of PostcopyBlocktimeContext will live till QEMU exit, but part of it (vcpu_addr, page_fault_vcpu_time) used only during calculation, will be released when postcopy ended or failed. To enable postcopy blocktime calculation on destination, need to request proper capabiltiy (Patch for documentation will be at the tail of the patch set). As an example following command enable that capability, assume QEMU was started with -chardev socket,id=charmonitor,path=/var/lib/migrate-vm-monitor.sock option to control it [root@host]#printf "{\"execute\" : \"qmp_capabilities\"}\r\n \ {\"execute\": \"migrate-set-capabilities\" , \"arguments\": { \"capabilities\": [ { \"capability\": \"postcopy-blocktime\", \"state\": true } ] } }" | nc -U /var/lib/migrate-vm-monitor.sock Or just with HMP (qemu) migrate_set_capability postcopy-blocktime on Signed-off-by: Alexey Perevalov --- include/migration/migration.h | 8 ++++++ migration/postcopy-ram.c | 65 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 73 insertions(+) diff --git a/include/migration/migration.h b/include/migration/migration.h index 2e61df5..766e802 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -49,6 +49,8 @@ enum mig_rp_message_type { MIG_RP_MSG_MAX }; +struct PostcopyBlocktimeContext; + /* State for the incoming migration */ struct MigrationIncomingState { QEMUFile *from_src_file; @@ -86,6 +88,12 @@ struct MigrationIncomingState { /* The coroutine we should enter (back) after failover */ Coroutine *migration_incoming_co; QemuSemaphore colo_incoming_sem; + + /* + * PostcopyBlocktimeContext to keep information for postcopy + * live migration, to calculate vCPU block time + * */ + struct PostcopyBlocktimeContext *blocktime_ctx; }; MigrationIncomingState *migration_incoming_get_current(void); diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index cbe8f9f..ade7f1c 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -63,6 +63,58 @@ struct PostcopyDiscardState { #include #include +typedef struct PostcopyBlocktimeContext { + /* time when page fault initiated per vCPU */ + int64_t *page_fault_vcpu_time; + /* page address per vCPU */ + uint64_t *vcpu_addr; + int64_t total_blocktime; + /* blocktime per vCPU */ + int64_t *vcpu_blocktime; + /* point in time when last page fault was initiated */ + int64_t last_begin; + /* number of vCPU are suspended */ + int smp_cpus_down; + + /* + * Handler for exit event, necessary for + * releasing whole blocktime_ctx + */ + Notifier exit_notifier; + /* + * Handler for postcopy event, necessary for + * releasing unnecessary part of blocktime_ctx + */ + Notifier postcopy_notifier; +} PostcopyBlocktimeContext; + +static void destroy_blocktime_context(struct PostcopyBlocktimeContext *ctx) +{ + g_free(ctx->page_fault_vcpu_time); + g_free(ctx->vcpu_addr); + g_free(ctx->vcpu_blocktime); + g_free(ctx); +} + +static void migration_exit_cb(Notifier *n, void *data) +{ + PostcopyBlocktimeContext *ctx = container_of(n, PostcopyBlocktimeContext, + exit_notifier); + destroy_blocktime_context(ctx); +} + +static struct PostcopyBlocktimeContext *blocktime_context_new(void) +{ + PostcopyBlocktimeContext *ctx = g_new0(PostcopyBlocktimeContext, 1); + ctx->page_fault_vcpu_time = g_new0(int64_t, smp_cpus); + ctx->vcpu_addr = g_new0(uint64_t, smp_cpus); + ctx->vcpu_blocktime = g_new0(int64_t, smp_cpus); + + ctx->exit_notifier.notify = migration_exit_cb; + qemu_add_exit_notifier(&ctx->exit_notifier); + add_migration_state_change_notifier(&ctx->postcopy_notifier); + return ctx; +} /** * receive_ufd_features: check userfault fd features, to request only supported @@ -155,6 +207,19 @@ static bool ufd_check_and_apply(int ufd, MigrationIncomingState *mis) } } +#ifdef UFFD_FEATURE_THREAD_ID + if (migrate_postcopy_blocktime() && mis && + UFFD_FEATURE_THREAD_ID & supported_features) { + /* kernel supports that feature */ + /* don't create blocktime_context if it exists */ + if (!mis->blocktime_ctx) { + mis->blocktime_ctx = blocktime_context_new(); + } + + asked_features |= UFFD_FEATURE_THREAD_ID; + } +#endif + /* * request features, even if asked_features is 0, due to * kernel expects UFFD_API before UFFDIO_REGISTER, per