From patchwork Mon Aug 27 01:32:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Wei W" X-Patchwork-Id: 10576349 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5ADC314E1 for ; Mon, 27 Aug 2018 02:02:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 488B3297B5 for ; Mon, 27 Aug 2018 02:02:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3BACE297B7; Mon, 27 Aug 2018 02:02:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AE315297B8 for ; Mon, 27 Aug 2018 02:02:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 12F8B6B3E07; Sun, 26 Aug 2018 22:02:09 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 113056B3E09; Sun, 26 Aug 2018 22:02:09 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F37FB6B3E08; Sun, 26 Aug 2018 22:02:08 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id AFD746B3E05 for ; Sun, 26 Aug 2018 22:02:08 -0400 (EDT) Received: by mail-pf1-f198.google.com with SMTP id h65-v6so10349426pfk.18 for ; Sun, 26 Aug 2018 19:02:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id; bh=9o3QnSG0QQc6+zV0+3KAl6vvz7AVHd2NNHKg5WtjcMI=; b=pE05ftLEPsqJaq1rXgqJODJrUdyRlwF3C6KQqmkI+dLRxQZcaYp4w72lf3NDhaCN+C cgzda6thzSCT4+/OFMHJ2t/Vaj1Id3uXBafVOtFinnDsOMYDgxONqZ4EfgQLQflJwTQU V39wv3MHb7s8ff6KWFHhzPh+nB/RaYBW6DLk/8o7yRoxFmvLdfoS54bxItLpgeilIINz j8Oph8zctQPNPkyBlADPrcTrL5A/disy4M/kCxfpXMOD0nDT1jCcfjAmzmM2Hea7u7nK FlEnLXNVxAUzJREYawZX60svDHTVus60jl3lc8m0I5BPCfQir+ZQNK9YAo8cniAXUcPW SN5w== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of wei.w.wang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=wei.w.wang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APzg51BqS2TB54ph41WpC3Yl1SObgRas/87NhvF8a31fDyvGmJZ/T03w S0nef8QfDPMv0DODpfOf8qKger5hZVog3hyiPG8RL3driz2i+q7HuTL9yHhs8oByOqKfxagkRFd 2T70uEKKdYVaqpDA2iPXtQowu4A8KTan6/OVaa9Jno0xHLosmfrJcqvm4aTRe/qwZGQ== X-Received: by 2002:a17:902:a5c2:: with SMTP id t2-v6mr11095116plq.85.1535335328361; Sun, 26 Aug 2018 19:02:08 -0700 (PDT) X-Google-Smtp-Source: ANB0Vdbhepeoh+dxldUhDnyuuufleXeL1T2FQJhWkSqo4PA+QppW0YJsJwP6ugGpnbtf8FIoalbI X-Received: by 2002:a17:902:a5c2:: with SMTP id t2-v6mr11095064plq.85.1535335327374; Sun, 26 Aug 2018 19:02:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535335327; cv=none; d=google.com; s=arc-20160816; b=sFApmeY/99K0ny0moOjs2pGuFB+9AJFLfR0mhs1UxIAmGInmtIiICuK5cZdfZCYSXC clL73/06p2/nMacU7BoI2bgRBkhHSLnCcYzfuC1e1w7vvwVZO8AqKgCFVNWo1IIMEICR d3Uv3+wEu8tdVkS+NLTQbCKsf5GJTU9hgW3DeERUEFvo/47T3KHEaWE5tQkgaUlE7jFS qVxapztwxPTt70NVIj/fGj2BdSjbzeLkhKGkBIrN/ZQisKySA4cpKXvxWxj91FmYKbPb q3VbHesqcx7PNlLOYu/hAQQkqYeXw+VaeBmw5VotZfjCLk8ZIRPzhcRHGjjcJqVF9IN8 FUlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from:arc-authentication-results; bh=9o3QnSG0QQc6+zV0+3KAl6vvz7AVHd2NNHKg5WtjcMI=; b=OIZ/6MbpNDueQ+yHBGQQw/mg5K1d+PJfYypxCbCfJ35Dx1JH1kraWVJG7N3W/ufvnE hPnwiOek2C1XMaOmAJn93p0yc/QwvflAhoXy/he8xbgTOhW143rnBiSKaCuGSadQI9Wy LCVVUEyDV5mmCVyQ4b9mZ7zBXOK2h41UpFgyaxWn3cDKHAGVLg3oA+GRlZi/BYKzZ262 X9va6aWpQ7M2Y698eVv6f3I4kLB7GexNGESOyfM06N+BPjd0wd8WFNZ/5xoMgDFLvHeQ vBmfXZ8MOPZImDnOGJKURpw3QsGR9ARUGrQXkbY+J+jvPodGJJEaShTCvAxsDS3eBax1 x27g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of wei.w.wang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=wei.w.wang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga18.intel.com (mga18.intel.com. [134.134.136.126]) by mx.google.com with ESMTPS id h17-v6si12964984pgj.214.2018.08.26.19.02.06 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 26 Aug 2018 19:02:07 -0700 (PDT) Received-SPF: pass (google.com: domain of wei.w.wang@intel.com designates 134.134.136.126 as permitted sender) client-ip=134.134.136.126; Authentication-Results: mx.google.com; spf=pass (google.com: domain of wei.w.wang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=wei.w.wang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Aug 2018 19:02:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,293,1531810800"; d="scan'208";a="75801307" Received: from devel-ww.sh.intel.com ([10.239.48.110]) by FMSMGA003.fm.intel.com with ESMTP; 26 Aug 2018 19:01:31 -0700 From: Wei Wang To: virtio-dev@lists.oasis-open.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, linux-mm@kvack.org, mst@redhat.com, mhocko@kernel.org, akpm@linux-foundation.org, dgilbert@redhat.com Cc: torvalds@linux-foundation.org, pbonzini@redhat.com, wei.w.wang@intel.com, liliang.opensource@gmail.com, yang.zhang.wz@gmail.com, quan.xu0@gmail.com, nilal@redhat.com, riel@redhat.com, peterx@redhat.com, quintela@redhat.com Subject: [PATCH v37 0/3] Virtio-balloon: support free page reporting Date: Mon, 27 Aug 2018 09:32:16 +0800 Message-Id: <1535333539-32420-1-git-send-email-wei.w.wang@intel.com> X-Mailer: git-send-email 2.7.4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.001181, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The new feature, VIRTIO_BALLOON_F_FREE_PAGE_HINT, implemented by this series enables the virtio-balloon driver to report hints of guest free pages to host. It can be used to accelerate virtual machine (VM) live migration. Here is an introduction of this usage: Live migration needs to transfer the VM's memory from the source machine to the destination round by round. For the 1st round, all the VM's memory is transferred. From the 2nd round, only the pieces of memory that were written by the guest (after the 1st round) are transferred. One method that is popularly used by the hypervisor to track which part of memory is written is to have the hypervisor write-protect all the guest memory. This feature enables the optimization by skipping the transfer of guest free pages during VM live migration. It is not concerned that the memory pages are used after they are given to the hypervisor as a hint of the free pages, because they will be tracked by the hypervisor and transferred in the subsequent round if they are used and written. * Tests 1 Test Environment Host: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz Migration setup: migrate_set_speed 100G, migrate_set_downtime 400ms 2 Test Results (results are averaged over several repeated runs) 2.1 Guest setup: 8G RAM, 4 vCPU 2.1.1 Idle guest live migration time Optimization v.s. Legacy = 620ms vs 2970ms --> ~79% reduction 2.1.2 Guest live migration with Linux compilation workload (i.e. make bzImage -j4) running 1) Live Migration Time: Optimization v.s. Legacy = 2273ms v.s. 4502ms --> ~50% reduction 2) Linux Compilation Time: Optimization v.s. Legacy = 8min42s v.s. 8min43s --> no obvious difference 2.2 Guest setup: 128G RAM, 4 vCPU 2.2.1 Idle guest live migration time Optimization v.s. Legacy = 5294ms vs 41651ms --> ~87% reduction 2.2.2 Guest live migration with Linux compilation workload 1) Live Migration Time: Optimization v.s. Legacy = 8816ms v.s. 54201ms --> 84% reduction 2) Linux Compilation Time: Optimization v.s. Legacy = 8min30s v.s. 8min36s --> no obvious difference ChangeLog: v36->v37: - free the reported pages to mm when receives a DONE cmd from host. Please see patch 1's commit log for reasons. Please see patch 1's commit for detailed explanations. For ChangeLogs from v22 to v36, please reference https://lkml.org/lkml/2018/7/20/199 For ChangeLogs before v21, please reference https://lwn.net/Articles/743660/ Wei Wang (3): virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT mm/page_poison: expose page_poisoning_enabled to kernel modules virtio-balloon: VIRTIO_BALLOON_F_PAGE_POISON drivers/virtio/virtio_balloon.c | 374 ++++++++++++++++++++++++++++++++---- include/uapi/linux/virtio_balloon.h | 8 + mm/page_poison.c | 6 + 3 files changed, 355 insertions(+), 33 deletions(-)