From patchwork Mon Oct 30 13:16:26 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Perevalov X-Patchwork-Id: 10032639 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B87B7603B4 for ; Mon, 30 Oct 2017 13:22:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A0F3928113 for ; Mon, 30 Oct 2017 13:22:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 95F1E288AD; Mon, 30 Oct 2017 13:22:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E790428113 for ; Mon, 30 Oct 2017 13:22:18 +0000 (UTC) Received: from localhost ([::1]:40698 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e9A1C-0005SZ-33 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 30 Oct 2017 09:22:18 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52366) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e99w3-0002GP-9I for qemu-devel@nongnu.org; Mon, 30 Oct 2017 09:17:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e99vx-00045I-8y for qemu-devel@nongnu.org; Mon, 30 Oct 2017 09:16:59 -0400 Received: from mailout2.w1.samsung.com ([210.118.77.12]:35964) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e99vw-00043W-VJ for qemu-devel@nongnu.org; Mon, 30 Oct 2017 09:16:53 -0400 Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20171030131649euoutp021196a2e3e127b1a27256846b3e0d0b03~yW3sfqmtN2108521085euoutp02j; Mon, 30 Oct 2017 13:16:49 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20171030131649euoutp021196a2e3e127b1a27256846b3e0d0b03~yW3sfqmtN2108521085euoutp02j DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1509369409; bh=IrbbQHw9BvIzYTUXQugsjBpcfl9GpY3OEIM+IoQK/YY=; h=From:To:Cc:Subject:Date:In-reply-to:References:From; b=Hc91WaHo6wbZnyICLJfOHXfSYXdBgPzRe3uaxFoHIZqXo9kmjQdxpxHmaczpcH3IX PvClui3OII5bV/EM1D8pbB0VJimSoz947/wInQaAmS1kGTnvlDedFg7kiagAnmG7Zh ffTnZVgHN8pwqrDpzFxDz0qZiFq3HERhyGgxvSN8= Received: from eusmges3.samsung.com (unknown [203.254.199.242]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20171030131648eucas1p2a5744c89703c557af8a057ed2ddbf682~yW3rsnvWv1096110961eucas1p2N; Mon, 30 Oct 2017 13:16:48 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges3.samsung.com (EUCPMTA) with SMTP id 96.4B.12867.04627F95; Mon, 30 Oct 2017 13:16:48 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20171030131647eucas1p1b5fb0616e683e78fdca767d267400f50~yW3rCHaBs1758017580eucas1p11; Mon, 30 Oct 2017 13:16:47 +0000 (GMT) X-AuditID: cbfec7f2-f793b6d000003243-de-59f72640bbeb Received: from eusync3.samsung.com ( [203.254.199.213]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id FE.23.20118.F3627F95; Mon, 30 Oct 2017 13:16:47 +0000 (GMT) Received: from aperevalov-ubuntu.rnd.samsung.ru ([106.109.129.199]) by eusync3.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0OYN00A7O0VNBHD0@eusync3.samsung.com>; Mon, 30 Oct 2017 13:16:47 +0000 (GMT) From: Alexey Perevalov To: qemu-devel@nongnu.org Date: Mon, 30 Oct 2017 16:16:26 +0300 Message-id: <1509369390-8285-3-git-send-email-a.perevalov@samsung.com> X-Mailer: git-send-email 2.7.4 In-reply-to: <1509369390-8285-1-git-send-email-a.perevalov@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrGIsWRmVeSWpSXmKPExsWy7djP87oOat8jDbq/i1nMvXuexaJ32z12 i2mfb7NbXGn/yW6xZf83dovjvTtYLO5s6WNyYPd4cm0zk8f7fVfZPPq2rGIMYI7isklJzcks Sy3St0vgyri+7T5bwTW1il1LtjE3MDbJdzFyckgImEgsmPmOFcIWk7hwbz1bFyMXh5DAUkaJ I8072CGcz4wSj6ceZoTpmLLyLDNEYhmjxK4TK5ggnG4midWrfwBVcXCwCRhI7LtnC9IgIiAp 8bvrNFgDs8BCRokp+5+zgSSEBWIkNkz6AzaVRUBV4vz+HrA4r4CbxPYpO6G2yUncPNfJDGJz CrhL3DrwAWyZhEAHm8S991PYIYpcJL5++gtlC0u8Or4FypaRuDy5mwWioZ1RontnJyuEM4FR 4sx0mA57iVM3rzKB2MwCfBKTtk1nBnlBQoBXoqNNCKLEQ+L/1RnsEGFHiWcX3SE+ngX0/pfJ 7BMYpRcwMqxiFEktLc5NTy021itOzC0uzUvXS87P3cQIjM3T/45/2sH49YTVIUYBDkYlHt4b ud8ihVgTy4orcw8xSnAwK4nwrlL8HinEm5JYWZValB9fVJqTWnyIUZqDRUmc1zaqLVJIID2x JDU7NbUgtQgmy8TBKdXA2PHx3FedaeklUp9EFJ8nG79fw+coeF4+zvfe6nlmcrIGF49aJNx2 lZVn1P3hfa6e2bK9Yzav5Dn/aPnu5Y8aFigI3FS9Zly47N4rsTyOAzlHFarO2za2x4uW/Oz4 WaIsL/yt7fmVjUKPLvY9vjZFznNuzfflhw5Ovrlqzk6mZewx96bfeeZ1W4mlOCPRUIu5qDgR AAGOaTLJAgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupiluLIzCtJLcpLzFFi42I5/e/4VV17te+RBvOazS3m3j3PYtG77R67 xbTPt9ktrrT/ZLfYsv8bu8Xx3h0sFne29DE5sHs8ubaZyeP9vqtsHn1bVjEGMEdx2aSk5mSW pRbp2yVwZVzfdp+t4Jpaxa4l25gbGJvkuxg5OSQETCSmrDzLDGGLSVy4t56ti5GLQ0hgCaPE 6QsT2SGcXiaJF3MusHYxcnCwCRhI7LtnC9IgIiAp8bvrNDNIDbPAQkaJS92vwSYJC8RIbJj0 hxHEZhFQlTi/v4cNxOYVcJPYPmUnI8Q2OYmb5zrB6jkF3CVuHfjABGILAdXM3n6BcQIj7wJG hlWMIqmlxbnpucVGesWJucWleel6yfm5mxiBIbTt2M8tOxi73gUfYhTgYFTi4b2R+y1SiDWx rLgy9xCjBAezkgjvKsXvkUK8KYmVValF+fFFpTmpxYcYpTlYlMR5e/esjhQSSE8sSc1OTS1I LYLJMnFwSjUwcmf775qwPU0+NsbT6W78GTXHbxa5D62Pbvykd8SHOfdOLbNMuNCe8r/Ol/+c 29l1qlksLNnSh3OF6rwZcjOVnk2VVl+wau7Ubxd9Prnf+j53msgj8a2q4necTuapXGeJiq1w aKiRstgQ96Zt2ecNz0XFVPXdV72JrW0XDH1/vCn0baPst+NzlFiKMxINtZiLihMBcdur4R0C AAA= X-CMS-MailID: 20171030131647eucas1p1b5fb0616e683e78fdca767d267400f50 X-Msg-Generator: CA CMS-TYPE: 201P X-CMS-RootMailID: 20171030131647eucas1p1b5fb0616e683e78fdca767d267400f50 X-RootMTR: 20171030131647eucas1p1b5fb0616e683e78fdca767d267400f50 References: <1509369390-8285-1-git-send-email-a.perevalov@samsung.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 210.118.77.12 Subject: [Qemu-devel] [PATCH v12 2/6] migration: add postcopy blocktime ctx into MigrationIncomingState X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: heetae82.ahn@samsung.com, quintela@redhat.com, dgilbert@redhat.com, peterx@redhat.com, Alexey Perevalov , i.maximets@samsung.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch adds request to kernel space for UFFD_FEATURE_THREAD_ID, in case when this feature is provided by kernel. PostcopyBlocktimeContext is incapsulated inside postcopy-ram.c, due to it's postcopy only feature. Also it defines PostcopyBlocktimeContext's instance live time. Information from PostcopyBlocktimeContext instance will be provided much after postcopy migration end, instance of PostcopyBlocktimeContext will live till QEMU exit, but part of it (vcpu_addr, page_fault_vcpu_time) used only during calculation, will be released when postcopy ended or failed. To enable postcopy blocktime calculation on destination, need to request proper capabiltiy (Patch for documentation will be at the tail of the patch set). As an example following command enable that capability, assume QEMU was started with -chardev socket,id=charmonitor,path=/var/lib/migrate-vm-monitor.sock option to control it [root@host]#printf "{\"execute\" : \"qmp_capabilities\"}\r\n \ {\"execute\": \"migrate-set-capabilities\" , \"arguments\": { \"capabilities\": [ { \"capability\": \"postcopy-blocktime\", \"state\": true } ] } }" | nc -U /var/lib/migrate-vm-monitor.sock Or just with HMP (qemu) migrate_set_capability postcopy-blocktime on Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Alexey Perevalov Reviewed-by: Juan Quintela --- migration/migration.h | 8 +++++++ migration/postcopy-ram.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 67 insertions(+) diff --git a/migration/migration.h b/migration/migration.h index 5f5e527..fb8d2ef 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -22,6 +22,8 @@ #include "hw/qdev.h" #include "io/channel.h" +struct PostcopyBlocktimeContext; + /* State for the incoming migration */ struct MigrationIncomingState { QEMUFile *from_src_file; @@ -59,6 +61,12 @@ struct MigrationIncomingState { /* The coroutine we should enter (back) after failover */ Coroutine *migration_incoming_co; QemuSemaphore colo_incoming_sem; + + /* + * PostcopyBlocktimeContext to keep information for postcopy + * live migration, to calculate vCPU block time + * */ + struct PostcopyBlocktimeContext *blocktime_ctx; }; MigrationIncomingState *migration_incoming_get_current(void); diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index bec6c2c..c18ec5a 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -61,6 +61,52 @@ struct PostcopyDiscardState { #include #include +typedef struct PostcopyBlocktimeContext { + /* time when page fault initiated per vCPU */ + int64_t *page_fault_vcpu_time; + /* page address per vCPU */ + uint64_t *vcpu_addr; + int64_t total_blocktime; + /* blocktime per vCPU */ + int64_t *vcpu_blocktime; + /* point in time when last page fault was initiated */ + int64_t last_begin; + /* number of vCPU are suspended */ + int smp_cpus_down; + + /* + * Handler for exit event, necessary for + * releasing whole blocktime_ctx + */ + Notifier exit_notifier; +} PostcopyBlocktimeContext; + +static void destroy_blocktime_context(struct PostcopyBlocktimeContext *ctx) +{ + g_free(ctx->page_fault_vcpu_time); + g_free(ctx->vcpu_addr); + g_free(ctx->vcpu_blocktime); + g_free(ctx); +} + +static void migration_exit_cb(Notifier *n, void *data) +{ + PostcopyBlocktimeContext *ctx = container_of(n, PostcopyBlocktimeContext, + exit_notifier); + destroy_blocktime_context(ctx); +} + +static struct PostcopyBlocktimeContext *blocktime_context_new(void) +{ + PostcopyBlocktimeContext *ctx = g_new0(PostcopyBlocktimeContext, 1); + ctx->page_fault_vcpu_time = g_new0(int64_t, smp_cpus); + ctx->vcpu_addr = g_new0(uint64_t, smp_cpus); + ctx->vcpu_blocktime = g_new0(int64_t, smp_cpus); + + ctx->exit_notifier.notify = migration_exit_cb; + qemu_add_exit_notifier(&ctx->exit_notifier); + return ctx; +} /** * receive_ufd_features: check userfault fd features, to request only supported @@ -153,6 +199,19 @@ static bool ufd_check_and_apply(int ufd, MigrationIncomingState *mis) } } +#ifdef UFFD_FEATURE_THREAD_ID + if (migrate_postcopy_blocktime() && mis && + UFFD_FEATURE_THREAD_ID & supported_features) { + /* kernel supports that feature */ + /* don't create blocktime_context if it exists */ + if (!mis->blocktime_ctx) { + mis->blocktime_ctx = blocktime_context_new(); + } + + asked_features |= UFFD_FEATURE_THREAD_ID; + } +#endif + /* * request features, even if asked_features is 0, due to * kernel expects UFFD_API before UFFDIO_REGISTER, per