From patchwork Thu Jan 9 14:57:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325885 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 116DA1398 for ; Thu, 9 Jan 2020 14:57:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D8B192072E for ; Thu, 9 Jan 2020 14:57:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="I4vxO4kv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731904AbgAIO5k (ORCPT ); Thu, 9 Jan 2020 09:57:40 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:42830 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728737AbgAIO5j (ORCPT ); Thu, 9 Jan 2020 09:57:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581858; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gDGiLHLaS/+Io2Phnnfgd/F37FeteIfvFFEQMfXM9RA=; b=I4vxO4kvES6dyJp5GiiSzsDObNjvHp0j4hy8RSvzfW0CN1JF9lxD593H9eC98Tk2Bhe/gZ +num04urfX6sZ0+n+oXUrIcUCHTRreqQDi11C1RZhsukR2gptTRjH+On0fcQYIfSdFQq7x MYdTeVVX2GjllxUqGUVwGKUHEgSe7hA= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-368-JBvgo9C2MTuTk0hsVIUiFg-1; Thu, 09 Jan 2020 09:57:37 -0500 X-MC-Unique: JBvgo9C2MTuTk0hsVIUiFg-1 Received: by mail-qv1-f71.google.com with SMTP id v5so4250765qvn.21 for ; Thu, 09 Jan 2020 06:57:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gDGiLHLaS/+Io2Phnnfgd/F37FeteIfvFFEQMfXM9RA=; b=H1zPXitV4HzVekxKQ/O3Kf9Oh70lxDhh0TKB4+2yB5e5dPqG7Gwl7zkT3aHJ4aqhvj fCI0AvM4iZT28LXfF/Q7krgOrkjIk6TJM9Eom2Hni9A0RGfNMqrK3jc15XmIEYVxL1tU boutkVjdi/LYeqSFvUrxgTQCW0xZyAoj8vWKdiDwqh4Y3Ko8x3xZ8vY+10h+Za+zl6Oc y+2FWr9bcPDp6A7JbxOopNgbpzttnsqxzRf91CdDToV/D/C3MkOB9X2BpHvo+r/fhOSZ KoTfjLF2boZd44Hvkh1NfdAg8dnePR6Pa8dEFw7o+FqN9o4UxgrzYWIA35+QQNfgNMKU AOJw== X-Gm-Message-State: APjAAAWd/TkpKehRjQaJ+U39ReTkiYvIYhxHVEYOzZLlXgCzNj/eSrpE 98aw9W2CUmQdKSCumZKp515TDXFBHwFX6TEGvPYoCS4Dt21qZ9Z66ICc1dTcgDpz0yVdwxwaDlf 3sM/l+nmlJaYS X-Received: by 2002:a05:620a:102e:: with SMTP id a14mr9510823qkk.159.1578581855559; Thu, 09 Jan 2020 06:57:35 -0800 (PST) X-Google-Smtp-Source: APXvYqw9fW22cpWNa0ej5q9Tvxem01vg3v97AmZT7K2M9MRr1qsTUoG9MFfgUMx1bfEecySbeqsNlQ== X-Received: by 2002:a05:620a:102e:: with SMTP id a14mr9510799qkk.159.1578581855290; Thu, 09 Jan 2020 06:57:35 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.57.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:57:33 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 01/21] vfio: introduce vfio_iova_rw to read/write a range of IOVAs Date: Thu, 9 Jan 2020 09:57:09 -0500 Message-Id: <20200109145729.32898-2-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Yan Zhao vfio_iova_rw will read/write a range of userspace memory (starting form device iova to iova + len -1) into a kenrel buffer without pinning the userspace memory. TODO: vfio needs to mark the iova dirty if vfio_iova_rw(write) is called. Cc: Kevin Tian Signed-off-by: Yan Zhao Signed-off-by: Peter Xu --- drivers/vfio/vfio.c | 45 ++++++++++++++++++ drivers/vfio/vfio_iommu_type1.c | 81 +++++++++++++++++++++++++++++++++ include/linux/vfio.h | 5 ++ 3 files changed, 131 insertions(+) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index c8482624ca34..36e91e647ed5 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1961,6 +1961,51 @@ int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, int npage) } EXPORT_SYMBOL(vfio_unpin_pages); +/* + * Read/Write a range of userspace IOVAs for a device into/from a kernel + * buffer without pinning the userspace memory + * @dev [in] : device + * @iova [in] : base IOVA of a userspace buffer + * @data [in] : pointer to kernel buffer + * @len [in] : kernel buffer length + * @write : indicate read or write + * Return error on failure or 0 on success. + */ +int vfio_iova_rw(struct device *dev, unsigned long iova, void *data, + unsigned long len, bool write) +{ + struct vfio_container *container; + struct vfio_group *group; + struct vfio_iommu_driver *driver; + int ret = 0; + + if (!dev || !data || len <= 0) + return -EINVAL; + + group = vfio_group_get_from_dev(dev); + if (!group) + return -ENODEV; + + ret = vfio_group_add_container_user(group); + if (ret) + goto out; + + container = group->container; + driver = container->iommu_driver; + + if (likely(driver && driver->ops->iova_rw)) + ret = driver->ops->iova_rw(container->iommu_data, + iova, data, len, write); + else + ret = -ENOTTY; + + vfio_group_try_dissolve_container(group); +out: + vfio_group_put(group); + return ret; +} +EXPORT_SYMBOL(vfio_iova_rw); + static int vfio_register_iommu_notifier(struct vfio_group *group, unsigned long *events, struct notifier_block *nb) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 2ada8e6cdb88..aee191077235 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -27,6 +27,7 @@ #include #include #include +#include #include #include #include @@ -2326,6 +2327,85 @@ static int vfio_iommu_type1_unregister_notifier(void *iommu_data, return blocking_notifier_chain_unregister(&iommu->notifier, nb); } +static int next_segment(unsigned long len, int offset) +{ + if (len > PAGE_SIZE - offset) + return PAGE_SIZE - offset; + else + return len; +} + +static int vfio_iommu_type1_rw_iova_seg(struct vfio_iommu *iommu, + unsigned long iova, void *data, + unsigned long seg_len, + unsigned long offset, + bool write) +{ + struct mm_struct *mm; + unsigned long vaddr; + struct vfio_dma *dma; + bool kthread = current->mm == NULL; + int ret = 0; + + dma = vfio_find_dma(iommu, iova, PAGE_SIZE); + if (!dma) + return -EINVAL; + + mm = get_task_mm(dma->task); + + if (!mm) + return -ENODEV; + + if (kthread) + use_mm(mm); + else if (current->mm != mm) { + ret = -EINVAL; + goto out; + } + + vaddr = dma->vaddr + iova - dma->iova + offset; + + ret = write ? __copy_to_user((void __user *)vaddr, + data, seg_len) : + __copy_from_user(data, (void __user *)vaddr, + seg_len); + if (ret) + ret = -EFAULT; + + if (kthread) + unuse_mm(mm); +out: + mmput(mm); + return ret; +} + +static int vfio_iommu_type1_iova_rw(void *iommu_data, unsigned long iova, + void *data, unsigned long len, bool write) +{ + struct vfio_iommu *iommu = iommu_data; + int offset = iova & ~PAGE_MASK; + int seg_len; + int ret = 0; + + iova = iova & PAGE_MASK; + + mutex_lock(&iommu->lock); + while ((seg_len = next_segment(len, offset)) > 0) { + ret = vfio_iommu_type1_rw_iova_seg(iommu, iova, data, + seg_len, offset, write); + if (ret) + break; + + offset = 0; + len -= seg_len; + data += seg_len; + iova += PAGE_SIZE; + } + + mutex_unlock(&iommu->lock); + return ret; +} + static const struct vfio_iommu_driver_ops vfio_iommu_driver_ops_type1 = { .name = "vfio-iommu-type1", .owner = THIS_MODULE, @@ -2338,6 +2418,7 @@ static const struct vfio_iommu_driver_ops vfio_iommu_driver_ops_type1 = { .unpin_pages = vfio_iommu_type1_unpin_pages, .register_notifier = vfio_iommu_type1_register_notifier, .unregister_notifier = vfio_iommu_type1_unregister_notifier, + .iova_rw = vfio_iommu_type1_iova_rw, }; static int __init vfio_iommu_type1_init(void) diff --git a/include/linux/vfio.h b/include/linux/vfio.h index e42a711a2800..7bf18a31bbcf 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -82,6 +82,8 @@ struct vfio_iommu_driver_ops { struct notifier_block *nb); int (*unregister_notifier)(void *iommu_data, struct notifier_block *nb); + int (*iova_rw)(void *iommu_data, unsigned long iova, + void *data, unsigned long len, bool write); }; extern int vfio_register_iommu_driver(const struct vfio_iommu_driver_ops *ops); @@ -107,6 +109,9 @@ extern int vfio_pin_pages(struct device *dev, unsigned long *user_pfn, extern int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, int npage); +extern int vfio_iova_rw(struct device *dev, unsigned long iova, void *data, + unsigned long len, bool write); + /* each type has independent events */ enum vfio_notify_type { VFIO_IOMMU_NOTIFY = 0, From patchwork Thu Jan 9 14:57:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325887 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0F3471398 for ; Thu, 9 Jan 2020 14:57:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D76132072E for ; Thu, 9 Jan 2020 14:57:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="T09uUC7c" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731916AbgAIO5n (ORCPT ); Thu, 9 Jan 2020 09:57:43 -0500 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:44718 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1731907AbgAIO5m (ORCPT ); Thu, 9 Jan 2020 09:57:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581861; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UEeMY0L9h1cGjmSmJRSArJW3guXbSNp3v29BbhovZjg=; b=T09uUC7c7tL31araNhoWHTcMFHWBGs3KGkJ/4MMkWVxAf+CEusC/0WHEawv+QPQTwb3spL WPxMN/WApnlUjT7x51tprTr17Y+1++tsRLYQhWMzichBiVu4VxNQzERK8+3SRTBRnxocjA lEB6cbemZk8PPYatgNeznZJHrO6nwO0= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-220-FpmiIt5IP4Clv4Go5iFMJQ-1; Thu, 09 Jan 2020 09:57:40 -0500 X-MC-Unique: FpmiIt5IP4Clv4Go5iFMJQ-1 Received: by mail-qk1-f197.google.com with SMTP id j16so4298488qkk.17 for ; Thu, 09 Jan 2020 06:57:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UEeMY0L9h1cGjmSmJRSArJW3guXbSNp3v29BbhovZjg=; b=fRky5DXwiAdKybilgiYfCsRJ2r5VBtL496ast0iK6tI95qtTHM2qV0a6XSI8TVFatz ApLXdGmrB9ehCfWAo3DJZEqXAUdOTcpiR4L4zzsh+Ym3uUCWyMBg4cJXjQjV0Juw2eM0 Dru6Cu1B5WSDhJ0s/F6c7axVLwb1Ryv29DB2p7GQyOUd2d0WoG+BggzKWtuRseTZ+fcC DeNUyJyDQKyN2XBhhqSRsXRFl4BEA25wHyCOilTH9UnoL6L+LbrS6X8DGI9ZDfr+b7WS Yq6hBdj2WeR8NBgDzfMVeML+evmElYL5ubm3SIgYRXsxedHc1YortP6mFcnSw4JqdU/E IMZQ== X-Gm-Message-State: APjAAAWINLQYBc5gejmhM9YsSwEMoJIX8O4C+R5dpdNzLQC4w1Xno+35 odZYCNpME+y4Z7Vlj9slU+VbNNmbA4avpF+GyrZHGsFqnRI9aGiTugUQy04nYs76j6vt6e+fjkK C8Bh4cjVnzY+j X-Received: by 2002:a37:9ed1:: with SMTP id h200mr10197430qke.390.1578581858652; Thu, 09 Jan 2020 06:57:38 -0800 (PST) X-Google-Smtp-Source: APXvYqxyTk3NXqX9G3kGKxoo4ojyuQTBdPfBxO21UKAMm33TtYTFwDUkwRRvpBGjneEqklTgj8BP9w== X-Received: by 2002:a37:9ed1:: with SMTP id h200mr10197404qke.390.1578581858408; Thu, 09 Jan 2020 06:57:38 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.57.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:57:36 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 02/21] drm/i915/gvt: subsitute kvm_read/write_guest with vfio_iova_rw Date: Thu, 9 Jan 2020 09:57:10 -0500 Message-Id: <20200109145729.32898-3-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Yan Zhao As a device model, it is better to read/write guest memory using vfio interface, so that vfio is able to maintain dirty info of device IOVAs. Compared to CPU side interfaces kvm_read/write_guest(), vfio_iova_rw() has ~600 cycles more overhead on average. ------------------------------------- | interface | avg cpu cycles | |-----------------------------------| | kvm_write_guest | 1546 | | ----------------------------------| | kvm_read_guest | 686 | |-----------------------------------| | vfio_iova_rw(w) | 2233 | |-----------------------------------| | vfio_iova_rw(r) | 1262 | ------------------------------------- Comparison of benchmarks scores are as blow: --------------------------------------------------------- | avg score | kvm_read/write_guest | vfio_iova_rw | --------------------------------------------------------- | Glmark2 | 1132 | 1138.2 | --------------------------------------------------------- | Lightsmark | 61.558 | 61.538 | |-------------------------------------------------------- | OpenArena | 142.77 | 136.6 | --------------------------------------------------------- | Heaven | 698 | 686.8 | -------------------------------------------------------- No obvious performance downgrade found. Cc: Kevin Tian Signed-off-by: Yan Zhao [peterx: pass in "write" to vfio_iova_rw(), suggested by Paolo] Signed-off-by: Peter Xu --- drivers/gpu/drm/i915/gvt/kvmgt.c | 25 ++++++------------------- 1 file changed, 6 insertions(+), 19 deletions(-) diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c index 3259a1fa69e1..5fb82f285b98 100644 --- a/drivers/gpu/drm/i915/gvt/kvmgt.c +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c @@ -1968,31 +1968,18 @@ static int kvmgt_rw_gpa(unsigned long handle, unsigned long gpa, void *buf, unsigned long len, bool write) { struct kvmgt_guest_info *info; - struct kvm *kvm; - int idx, ret; - bool kthread = current->mm == NULL; + int ret; + struct intel_vgpu *vgpu; + struct device *dev; if (!handle_valid(handle)) return -ESRCH; info = (struct kvmgt_guest_info *)handle; - kvm = info->kvm; - - if (kthread) { - if (!mmget_not_zero(kvm->mm)) - return -EFAULT; - use_mm(kvm->mm); - } - - idx = srcu_read_lock(&kvm->srcu); - ret = write ? kvm_write_guest(kvm, gpa, buf, len) : - kvm_read_guest(kvm, gpa, buf, len); - srcu_read_unlock(&kvm->srcu, idx); + vgpu = info->vgpu; + dev = mdev_dev(vgpu->vdev.mdev); - if (kthread) { - unuse_mm(kvm->mm); - mmput(kvm->mm); - } + ret = vfio_iova_rw(dev, gpa, buf, len, write); return ret; } From patchwork Thu Jan 9 14:57:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325889 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7822C1398 for ; Thu, 9 Jan 2020 14:57:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5664C2080D for ; Thu, 9 Jan 2020 14:57:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="FnNcQcpw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731930AbgAIO5q (ORCPT ); Thu, 9 Jan 2020 09:57:46 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:46056 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1731918AbgAIO5q (ORCPT ); Thu, 9 Jan 2020 09:57:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581864; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TtlFeY7HzyndokCyv32BVKkuZLgrN4WwINmZ03zf3D4=; b=FnNcQcpw6h+jf99h/piGPGq0KMQQrlSUrgE+BkQyGzIkj1CKNe5uhIhxEvbQbVAng8BZrA 3PwwnTVLzTpubUxQzRY/Bg+tPPpSUNAKbwcZFRIiGy4chBgtwU7S3nLBfsWsbTS5H3lYZM W60h3XnBZS94aDaQh9WAIyV/lnrcg1U= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-260-m6icWMY9NiqVtbVBXcEf_w-1; Thu, 09 Jan 2020 09:57:43 -0500 X-MC-Unique: m6icWMY9NiqVtbVBXcEf_w-1 Received: by mail-qt1-f198.google.com with SMTP id m30so4364525qtb.2 for ; Thu, 09 Jan 2020 06:57:43 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TtlFeY7HzyndokCyv32BVKkuZLgrN4WwINmZ03zf3D4=; b=dzKGmsTdk1zbfGqbQem7kKbSpGVLsGcLKiKCPAEh1rvZ6+171c+nHVkNzd2Et66ySs eD7/gX4dDCfzP+ZQG8U0TyvgfGx8hmpbqY2nihO5I9CX3p1DBijJt9uT5dOGbj7nMJOV QB+N7YZHfEkZUvYJPkyPjNS2Yfcwp57cuJycCIFSZtzixVcs/MKDEZ6FJV9Kj4rEiupR 91UWLErwdUm+bBziZMup5t8E/QWP1/cm1V9KKm5vPRTABfMwqULHMGP114j/k20fCc3h z6HmXcqqYoXs5N6tuhFHZpxNx3Uu2Nz4dXg/eJfuvyd92axX3vJNHDbGhHwdmM6/j+HS OZOA== X-Gm-Message-State: APjAAAUz6N3HpPuCcWel+8ZVv/tmx1MiOZJnGgd395Zc3L+qBNalT0Xk ij0chrEV2k6dR3z61jBFfZG9OMWDx59Fp+uZQvqrWKaDVLC0/UKhrVvW+Vd9h5ybgtDiGEwSASV zODS2xUxrOond X-Received: by 2002:a37:52d5:: with SMTP id g204mr10243293qkb.215.1578581862081; Thu, 09 Jan 2020 06:57:42 -0800 (PST) X-Google-Smtp-Source: APXvYqwGbU0IaY9Ud+rTAWvjCd+8O9eQGTqr/fi0Y9HIplgcF1WCujVwGE+E2M6bfObjr+FYycncdQ== X-Received: by 2002:a37:52d5:: with SMTP id g204mr10243271qkb.215.1578581861864; Thu, 09 Jan 2020 06:57:41 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.57.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:57:40 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 03/21] KVM: Remove kvm_read_guest_atomic() Date: Thu, 9 Jan 2020 09:57:11 -0500 Message-Id: <20200109145729.32898-4-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Remove kvm_read_guest_atomic() because it's not used anywhere. Signed-off-by: Peter Xu --- include/linux/kvm_host.h | 2 -- virt/kvm/kvm_main.c | 11 ----------- 2 files changed, 13 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 528ab7a814ab..2337f9b6112c 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -725,8 +725,6 @@ void kvm_get_pfn(kvm_pfn_t pfn); int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset, int len); -int kvm_read_guest_atomic(struct kvm *kvm, gpa_t gpa, void *data, - unsigned long len); int kvm_read_guest(struct kvm *kvm, gpa_t gpa, void *data, unsigned long len); int kvm_read_guest_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, void *data, unsigned long len); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 3aa21bec028d..24c9cf4c8a52 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2048,17 +2048,6 @@ static int __kvm_read_guest_atomic(struct kvm_memory_slot *slot, gfn_t gfn, return 0; } -int kvm_read_guest_atomic(struct kvm *kvm, gpa_t gpa, void *data, - unsigned long len) -{ - gfn_t gfn = gpa >> PAGE_SHIFT; - struct kvm_memory_slot *slot = gfn_to_memslot(kvm, gfn); - int offset = offset_in_page(gpa); - - return __kvm_read_guest_atomic(slot, gfn, data, offset, len); -} -EXPORT_SYMBOL_GPL(kvm_read_guest_atomic); - int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu, gpa_t gpa, void *data, unsigned long len) { From patchwork Thu Jan 9 14:57:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325891 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B90A1398 for ; Thu, 9 Jan 2020 14:57:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 092FE2072E for ; Thu, 9 Jan 2020 14:57:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bUBu4NfT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731942AbgAIO5v (ORCPT ); Thu, 9 Jan 2020 09:57:51 -0500 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:35330 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1731932AbgAIO5u (ORCPT ); Thu, 9 Jan 2020 09:57:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581869; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Nm8O4Xii0OuE4zFVyte5k6dYfDYh4EMQdTOkPftgaqU=; b=bUBu4NfTvNOwNeOlih9btuxvpeY9H6uKq2FmwrA0QYPDVUNARjU81YtW1s1sLSyf2CyV75 WupAccSjwWd1DbO1i8O1+rF3u6RKGwxrrOVL2WcssDZbaKQRiTgnI4yP7Q0zFMQFeeCxar rA+Gp/Npm5Yl1Qx4dwyJF79XONEJ66Q= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-137-quoVFkJcOWi9IAEy97b2hg-1; Thu, 09 Jan 2020 09:57:46 -0500 X-MC-Unique: quoVFkJcOWi9IAEy97b2hg-1 Received: by mail-qk1-f200.google.com with SMTP id n128so4298116qke.19 for ; Thu, 09 Jan 2020 06:57:45 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Nm8O4Xii0OuE4zFVyte5k6dYfDYh4EMQdTOkPftgaqU=; b=b1J+gK/4qaD3OjuzY6vP04irX+yQrPB/9ZA7808lzrlc3y6sBdoZTb5P3riVOVPYwW wO8NguCGFQhHYJVojzEBgfUssG1XiFssT4mrR/p4cQgtGuYSxsHWXDYgNohv9cgzYltR eRsKmKK1BpV66ksS+vKulHwKG9vxscGuGOy5dWmDVpGGvWHPS36+o1/KvsadJaSSuwx1 uQOjj+kBLyVDt1D92QA43vEH6WOvfo54ykVB/8l/Rftp0DQyxnl91cAlo13LpB71IaL/ ic9pum4flrgH3xGE5zaz4M4wlpCEHeZhOCsqp4wmUpo3igt0CRd5Wk4oT+bBnzX6lR/Z j+mw== X-Gm-Message-State: APjAAAUrf/4NE+lD0GsL14yxMmB8ebnx/0/aLpos9yboqJpYLPHYM2PT EWPaKu9yo6vuBXAJFlKgZ9zIDm77IJitpjZODfhJrUAwmemu/LtuAdGctDgq2uXo10QP8kJz/NY 68PEE4hc1nins X-Received: by 2002:ae9:ec01:: with SMTP id h1mr10129989qkg.33.1578581864674; Thu, 09 Jan 2020 06:57:44 -0800 (PST) X-Google-Smtp-Source: APXvYqzNTAs/SAnbPG+ivO60ziyJlazNT+NDxUAFeemJ1UyIi2OiGROdyCVheaB21amr6IgdyN2YNA== X-Received: by 2002:ae9:ec01:: with SMTP id h1mr10129964qkg.33.1578581864482; Thu, 09 Jan 2020 06:57:44 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.57.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:57:43 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 04/21] KVM: Add build-time error check on kvm_run size Date: Thu, 9 Jan 2020 09:57:12 -0500 Message-Id: <20200109145729.32898-5-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org It's already going to reach 2400 Bytes (which is over half of page size on 4K page archs), so maybe it's good to have this build-time check in case it overflows when adding new fields. Signed-off-by: Peter Xu --- virt/kvm/kvm_main.c | 1 + 1 file changed, 1 insertion(+) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 24c9cf4c8a52..70b78ccaf3b5 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -338,6 +338,7 @@ int kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id) vcpu->pre_pcpu = -1; INIT_LIST_HEAD(&vcpu->blocked_vcpu_list); + BUILD_BUG_ON(sizeof(struct kvm_run) > PAGE_SIZE); page = alloc_page(GFP_KERNEL | __GFP_ZERO); if (!page) { r = -ENOMEM; From patchwork Thu Jan 9 14:57:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325893 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 893DC1580 for ; Thu, 9 Jan 2020 14:57:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 66FCB2072A for ; Thu, 9 Jan 2020 14:57:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ifvsF29A" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731952AbgAIO5y (ORCPT ); Thu, 9 Jan 2020 09:57:54 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:24932 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1731932AbgAIO5w (ORCPT ); Thu, 9 Jan 2020 09:57:52 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581871; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aRATN7sKWcZdWDij4U02uLG0iob0fByx2e0Dt2ueT/w=; b=ifvsF29AAoPaNiAHGjmliWV+BCppV20kafRoHF6alACjBXV07bNOtoAzxGDXao2nxE2uSi iU6iq9Xa5yRnDTFiH3vuPbizJJjwZfw+u64FmA7Z8GVnJSz1zw1FqfNt6FO6Ai9bVxdZzE 1Ci4jtQ1pGzBw4wOHGmqJsAF3OFmwIc= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-424-KWH8Ato6Ntq7kt7gq9iLGQ-1; Thu, 09 Jan 2020 09:57:50 -0500 X-MC-Unique: KWH8Ato6Ntq7kt7gq9iLGQ-1 Received: by mail-qv1-f71.google.com with SMTP id r9so4238977qvs.19 for ; Thu, 09 Jan 2020 06:57:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aRATN7sKWcZdWDij4U02uLG0iob0fByx2e0Dt2ueT/w=; b=I4BuhdGxmI4+5HPYH3a2oOW8v4ubY1mlF5P0VQ/xNKi+188shBsRllP9Ix87mcAMXQ +92KMzhbSo+uQnLZS7sfKYd0vjBrZMW55jaDKPOejaen4MlNWLAPDUKFB2/nOROjVmw6 nrz6snYvRxmE/XwNjuuSyX3m4Xhx+diB9utpo6aGHwsHkK/X+yMZCvR16N8CMj221DHp M6UBjgu4QSS+8PC+Ed9FNPtZWI97AouZqWznxuQ8brQ2le/nxQEPEMC/MB/5Uk6U9Dpc Jm68ucC8cNjjMbOoxOyfBOzDz2Fxa0J4BC5OcixBwmLFrbFWw3CiAIgekhlZbXENvn8I /pZw== X-Gm-Message-State: APjAAAVHD7LwWZJXzrrXkdWI67KflxGJEbVe6ED/g08qxNnbVU8PtAzd 8fLW+xIcvCcT3mKGnJXx72pNTG8dRxZylxo9mt+j6zZpg66yhRvaj9JZicUKH6bCD+R6SYsNPQk AZTLJPwySHVt+ X-Received: by 2002:a05:620a:102e:: with SMTP id a14mr9511643qkk.159.1578581868294; Thu, 09 Jan 2020 06:57:48 -0800 (PST) X-Google-Smtp-Source: APXvYqyRbVnZp1vzZ6tL+7JcD0WsY8mA3XV8L6qTwZsx0HCnJl+m4kcc+RK3EtswtsJg3qtmZCifYA== X-Received: by 2002:a05:620a:102e:: with SMTP id a14mr9511621qkk.159.1578581868090; Thu, 09 Jan 2020 06:57:48 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.57.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:57:46 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 05/21] KVM: X86: Change parameter for fast_page_fault tracepoint Date: Thu, 9 Jan 2020 09:57:13 -0500 Message-Id: <20200109145729.32898-6-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org It would be clearer to dump the return value to know easily on whether did we go through the fast path for handling current page fault. Remove the old two last parameters because after all the old/new sptes were dumped in the same line. Signed-off-by: Peter Xu --- arch/x86/kvm/mmutrace.h | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/mmutrace.h b/arch/x86/kvm/mmutrace.h index 3c6522b84ff1..456371406d2a 100644 --- a/arch/x86/kvm/mmutrace.h +++ b/arch/x86/kvm/mmutrace.h @@ -244,9 +244,6 @@ TRACE_EVENT( __entry->access) ); -#define __spte_satisfied(__spte) \ - (__entry->retry && is_writable_pte(__entry->__spte)) - TRACE_EVENT( fast_page_fault, TP_PROTO(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u32 error_code, @@ -274,12 +271,10 @@ TRACE_EVENT( ), TP_printk("vcpu %d gva %llx error_code %s sptep %p old %#llx" - " new %llx spurious %d fixed %d", __entry->vcpu_id, + " new %llx ret %d", __entry->vcpu_id, __entry->cr2_or_gpa, __print_flags(__entry->error_code, "|", kvm_mmu_trace_pferr_flags), __entry->sptep, - __entry->old_spte, __entry->new_spte, - __spte_satisfied(old_spte), __spte_satisfied(new_spte) - ) + __entry->old_spte, __entry->new_spte, __entry->retry) ); TRACE_EVENT( From patchwork Thu Jan 9 14:57:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325925 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A058E92A for ; Thu, 9 Jan 2020 14:59:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7EA742080D for ; Thu, 9 Jan 2020 14:59:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ZitsTMB9" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731960AbgAIO5z (ORCPT ); Thu, 9 Jan 2020 09:57:55 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:46961 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1731950AbgAIO5y (ORCPT ); Thu, 9 Jan 2020 09:57:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581873; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iOwBnRfuU+D9l17RCY4C+mlVimy3V5B0lUjjJ1K6zlg=; b=ZitsTMB9GqxN0L3wJGTl0fI5mP3dX9G96cl6ilpEZLfJAouO0RK39QzzMNwUvbuYKIivFI FFlIyoU5Axm2IroNrrFnWV+2RQCfj5+jf0ZfdwN+uLsayHcs4/m1m6N/+r9F5YmmZl2i57 Nruvdx/Wc+Kv71USSE9L5jg+ZzFwIgo= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-252-QN9pm6e6NfmaKMxYxZt3HA-1; Thu, 09 Jan 2020 09:57:52 -0500 X-MC-Unique: QN9pm6e6NfmaKMxYxZt3HA-1 Received: by mail-qk1-f197.google.com with SMTP id k10so4299100qki.2 for ; Thu, 09 Jan 2020 06:57:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iOwBnRfuU+D9l17RCY4C+mlVimy3V5B0lUjjJ1K6zlg=; b=ov8agg6DUVLP3Sq3Qan04j6hV8GoOC70YkpA4Drumf0ud7Qgl6FmJibFuhHQC8SsWG scm1lp2XZwco8BUbxROKc6WeZQBRyTb6bZ2wW0PNGybrKcPHFjm8L4XXh8LqsvUAOU0x a6jBaNm27qZ+14PK2m63ZeOnKvzKxzu0jpBE7VqiNHKu13V7+gk9BRZJU3wzTXkt+BRm eVf24VM4VAl8j/kf44Udglpx3qhFTakhkw2iwHFWUKB9hA3a2JQUHx1rbco0KKPGDd3j US+y0SKEpHoa9cJyFbe/LEzM0pOJxe+A7znyqBo6nCYeiVUoDHebn+XECWrFpXYpjDA/ RYFA== X-Gm-Message-State: APjAAAWVkyX7bCdxtoB4SY9yWi8qotuETc5svrarcE4lg8oaTEKhpzNn dm4dOeEpiiN2NVG+RupNi4C4RSTtXNG8zLArdsq8kgBBeKxPnHv4vw4fHQ4lWsiyS0YxBNMz2W4 6jRh+TbIIb/HZ X-Received: by 2002:a05:6214:903:: with SMTP id dj3mr9256932qvb.27.1578581871239; Thu, 09 Jan 2020 06:57:51 -0800 (PST) X-Google-Smtp-Source: APXvYqyjzq+Zf7itjTFb9hX/DEDGZiDvjZ55SpaSIA4PVTPWagLi4h5EO3WDSjPMFAcJ3UFk0noDsw== X-Received: by 2002:a05:6214:903:: with SMTP id dj3mr9256894qvb.27.1578581870928; Thu, 09 Jan 2020 06:57:50 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.57.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:57:50 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 06/21] KVM: X86: Don't take srcu lock in init_rmode_identity_map() Date: Thu, 9 Jan 2020 09:57:14 -0500 Message-Id: <20200109145729.32898-7-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org We've already got the slots_lock, so we should be safe. Signed-off-by: Peter Xu --- arch/x86/kvm/vmx/vmx.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index b5a0c2e05825..7add2fc8d8e9 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -3475,7 +3475,7 @@ static int init_rmode_tss(struct kvm *kvm) static int init_rmode_identity_map(struct kvm *kvm) { struct kvm_vmx *kvm_vmx = to_kvm_vmx(kvm); - int i, idx, r = 0; + int i, r = 0; kvm_pfn_t identity_map_pfn; u32 tmp; @@ -3483,7 +3483,7 @@ static int init_rmode_identity_map(struct kvm *kvm) mutex_lock(&kvm->slots_lock); if (likely(kvm_vmx->ept_identity_pagetable_done)) - goto out2; + goto out; if (!kvm_vmx->ept_identity_map_addr) kvm_vmx->ept_identity_map_addr = VMX_EPT_IDENTITY_PAGETABLE_ADDR; @@ -3492,9 +3492,8 @@ static int init_rmode_identity_map(struct kvm *kvm) r = __x86_set_memory_region(kvm, IDENTITY_PAGETABLE_PRIVATE_MEMSLOT, kvm_vmx->ept_identity_map_addr, PAGE_SIZE); if (r < 0) - goto out2; + goto out; - idx = srcu_read_lock(&kvm->srcu); r = kvm_clear_guest_page(kvm, identity_map_pfn, 0, PAGE_SIZE); if (r < 0) goto out; @@ -3510,9 +3509,6 @@ static int init_rmode_identity_map(struct kvm *kvm) kvm_vmx->ept_identity_pagetable_done = true; out: - srcu_read_unlock(&kvm->srcu, idx); - -out2: mutex_unlock(&kvm->slots_lock); return r; } From patchwork Thu Jan 9 14:57:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325923 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EC219109A for ; Thu, 9 Jan 2020 14:59:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CA5582072E for ; Thu, 9 Jan 2020 14:59:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XoqMRlwK" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732045AbgAIO73 (ORCPT ); Thu, 9 Jan 2020 09:59:29 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:46455 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1731964AbgAIO54 (ORCPT ); Thu, 9 Jan 2020 09:57:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581875; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EWV08i+y09T0shEQm4Du/cNi7j67EsBakky7HQM3iHM=; b=XoqMRlwKNuy1kh5HFK3GTjotAYxQSPD6ZXUydHAmmbS5nTouHc3aO0UhhmL2lxbCrqXzRQ XTsasPjEujaErIKF9NXOldLlSyatvLsZoRNPa2taTmHaFagqQWLN33XeV0S+XgNLt89pCS JxhWH/w78y+k6pZb+4JOizG3QveqVDE= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-407-n_dCF6DSOI2xKB_7FwZoMA-1; Thu, 09 Jan 2020 09:57:54 -0500 X-MC-Unique: n_dCF6DSOI2xKB_7FwZoMA-1 Received: by mail-qk1-f197.google.com with SMTP id c202so4323500qkg.4 for ; Thu, 09 Jan 2020 06:57:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EWV08i+y09T0shEQm4Du/cNi7j67EsBakky7HQM3iHM=; b=UTgiEgbkKHi4kv47xAgjV6+FlJD+KYlsViMkrdfTOoigQgXNCDa5ncKievfco9HB0e efrIApwTJV5t7SpQZu44VrvS74leqeH52SVASH/3lpe4cUHnt35yfA2AAJIMAMbh2VW5 gt5tfFdMWhmWQbdYoJv9iy5Yr+XK2DZY9Vz4uSLn8hwHD93lBCWSMw4HKzN8PnDWtjLQ GC8VG/LYs5VtncOrMBiYDtleL5cbv07m0l5QGlLZp/nKPnotRYss2T0a/kdz1TC49erh yYh6GBzSCDcqkqc2VCk5ZaRb2M1ASjylPwSJU8B1Ztt54a9TpT0O+JKJDvk2ZN4Sq1JM 02mg== X-Gm-Message-State: APjAAAU7XEAoaMTkFB13lr4WkBfuVI2i1JEw4CBvG6Fx5rdFXQw6kmP7 qM4pmTdoIk+FDm/WeVTJ7sekxBfxDlSKxoLkRqd1QM/w3Lz9Ka/PMp+r5A4rMdfzsAfLy61A+cN rNQeUhV0GtqFw X-Received: by 2002:a05:620a:136e:: with SMTP id d14mr9774132qkl.342.1578581873443; Thu, 09 Jan 2020 06:57:53 -0800 (PST) X-Google-Smtp-Source: APXvYqxCvoe5Ap6+5M5ORKV7xt2P5nsc1tcHZco1QoU+FUiuKNxmacCYbTCTEN1X9W8lMsTu+hLsCQ== X-Received: by 2002:a05:620a:136e:: with SMTP id d14mr9774109qkl.342.1578581873235; Thu, 09 Jan 2020 06:57:53 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.57.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:57:52 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 07/21] KVM: Cache as_id in kvm_memory_slot Date: Thu, 9 Jan 2020 09:57:15 -0500 Message-Id: <20200109145729.32898-8-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Cache the address space ID just like the slot ID. It will be used in order to fill in the dirty ring entries. Suggested-by: Paolo Bonzini Suggested-by: Sean Christopherson Signed-off-by: Peter Xu --- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 2337f9b6112c..763adf8c47b0 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -348,6 +348,7 @@ struct kvm_memory_slot { unsigned long userspace_addr; u32 flags; short id; + u8 as_id; }; static inline unsigned long kvm_dirty_bitmap_bytes(struct kvm_memory_slot *memslot) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 70b78ccaf3b5..1fd204f27028 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1040,6 +1040,8 @@ int __kvm_set_memory_region(struct kvm *kvm, new = old = *slot; + BUILD_BUG_ON(U8_MAX < KVM_ADDRESS_SPACE_NUM); + new.as_id = as_id; new.id = id; new.base_gfn = base_gfn; new.npages = npages; From patchwork Thu Jan 9 14:57:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325921 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 944DE92A for ; Thu, 9 Jan 2020 14:59:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 72D282072E for ; Thu, 9 Jan 2020 14:59:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ZWqJk9p3" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731981AbgAIO57 (ORCPT ); Thu, 9 Jan 2020 09:57:59 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:24545 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1731943AbgAIO56 (ORCPT ); Thu, 9 Jan 2020 09:57:58 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581877; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lpz/17PtQ4/jo6RMA0hys9EClp6/SYcaGKe9OUWI9XM=; b=ZWqJk9p3j4Nqz+4yGezcqL56XcH58XPlGS992FG3eA9XNWp9rzYUiCHmBFikUEm2EtB0lS 8Rc+frqxZrjYEoljLXjfppAsCHtNuY2RZj6n7wqT6YdmVZPT7pDKN2Nf0S1vArsTlqBLRU wA2KxK7dDPLMcndv6vrhPGsve6TxnPM= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-101-1FIyNC6LMfOy9x46OgfBbg-1; Thu, 09 Jan 2020 09:57:56 -0500 X-MC-Unique: 1FIyNC6LMfOy9x46OgfBbg-1 Received: by mail-qv1-f71.google.com with SMTP id e26so4266402qvb.4 for ; Thu, 09 Jan 2020 06:57:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Lpz/17PtQ4/jo6RMA0hys9EClp6/SYcaGKe9OUWI9XM=; b=KPSj5Vc5lqJY/bgnZq+etJk6soNRSSXTFxXckqjEuSj+twXxxW2LNvzuAhWBn1v4uz 97znKuDjtcLajR+zVgSMUy0cPwaAdSo05pVckpFyhNB03P8+fQhNdHS3MmRHHhjyj8Df 8Ba0NSWUJktIqRMSMUIRAIcZ2aJuHbhf4YZ0OHTGWa3dUvlgrHuNBy7mlxY/vSkVkC61 DQZ8p6yisSEW5uGHFkKYer5YAPV+pk1vMG14iuzUovYsDBBsvJWa9MhCGqp/fYt9wBan u18kbH21dnMkqZJ+JWfcAb0jb1Y9BPQ/yn0ZVJONG1NqNj7LtzNLEKcLO/fizfqkQdrK jZ+Q== X-Gm-Message-State: APjAAAWGZ7ne6og+OSkMywvSzxAQtRvQVc71xkPEO9qAXWOl+Q5igt9i ShaNPYE31zNIMbEzPSZtc+jLxrlr5nlrJDYQCS190Y9q20++yX+sAXAVXSFgr4CMRXLEmQISoJC md5ZEE84azbCZ X-Received: by 2002:a37:c24b:: with SMTP id j11mr9504200qkm.57.1578581875657; Thu, 09 Jan 2020 06:57:55 -0800 (PST) X-Google-Smtp-Source: APXvYqxfkz13asTRa6kA2E83DYCU0f8pOkw4MCnJqXEqAOZ8Ss3NtGdYvm9Ysn6NhuOv0a4mxrMWfQ== X-Received: by 2002:a37:c24b:: with SMTP id j11mr9504175qkm.57.1578581875412; Thu, 09 Jan 2020 06:57:55 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.57.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:57:54 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 08/21] KVM: X86: Drop x86_set_memory_region() Date: Thu, 9 Jan 2020 09:57:16 -0500 Message-Id: <20200109145729.32898-9-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The helper x86_set_memory_region() is only used in vmx_set_tss_addr() and kvm_arch_destroy_vm(). Push the lock upper in both cases. With that, drop x86_set_memory_region(). This prepares to allow __x86_set_memory_region() to return a HVA mapped, because the HVA will need to be protected by the lock too even after __x86_set_memory_region() returns. Signed-off-by: Peter Xu --- arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/vmx/vmx.c | 7 +++++-- arch/x86/kvm/x86.c | 22 +++++++--------------- 3 files changed, 12 insertions(+), 18 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 159a28512e4c..eb6673c7d2e3 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1619,7 +1619,6 @@ void __kvm_request_immediate_exit(struct kvm_vcpu *vcpu); int kvm_is_in_guest(void); int __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size); -int x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size); bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu); bool kvm_vcpu_is_bsp(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 7add2fc8d8e9..7e3d370209e0 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -4482,8 +4482,11 @@ static int vmx_set_tss_addr(struct kvm *kvm, unsigned int addr) if (enable_unrestricted_guest) return 0; - ret = x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, addr, - PAGE_SIZE * 3); + mutex_lock(&kvm->slots_lock); + ret = __x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, addr, + PAGE_SIZE * 3); + mutex_unlock(&kvm->slots_lock); + if (ret) return ret; to_kvm_vmx(kvm)->tss_addr = addr; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 93bbbce67a03..c4d3972dcd14 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9636,18 +9636,6 @@ int __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size) } EXPORT_SYMBOL_GPL(__x86_set_memory_region); -int x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size) -{ - int r; - - mutex_lock(&kvm->slots_lock); - r = __x86_set_memory_region(kvm, id, gpa, size); - mutex_unlock(&kvm->slots_lock); - - return r; -} -EXPORT_SYMBOL_GPL(x86_set_memory_region); - void kvm_arch_pre_destroy_vm(struct kvm *kvm) { kvm_mmu_pre_destroy_vm(kvm); @@ -9661,9 +9649,13 @@ void kvm_arch_destroy_vm(struct kvm *kvm) * unless the the memory map has changed due to process exit * or fd copying. */ - x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT, 0, 0); - x86_set_memory_region(kvm, IDENTITY_PAGETABLE_PRIVATE_MEMSLOT, 0, 0); - x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, 0, 0); + mutex_lock(&kvm->slots_lock); + __x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT, + 0, 0); + __x86_set_memory_region(kvm, IDENTITY_PAGETABLE_PRIVATE_MEMSLOT, + 0, 0); + __x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, 0, 0); + mutex_unlock(&kvm->slots_lock); } if (kvm_x86_ops->vm_destroy) kvm_x86_ops->vm_destroy(kvm); From patchwork Thu Jan 9 14:57:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325895 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 31DA292A for ; Thu, 9 Jan 2020 14:58:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 06EF02072A for ; Thu, 9 Jan 2020 14:58:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ijTPfaoh" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732006AbgAIO6G (ORCPT ); Thu, 9 Jan 2020 09:58:06 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:35507 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1731998AbgAIO6C (ORCPT ); Thu, 9 Jan 2020 09:58:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581880; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4C6ZGVemG6J37mDpNjuVXzVMZmBhm9r1bGAW/2NgwDc=; b=ijTPfaohnlOLDFHQrL/RR3JEBoid/jInEWQocL9X9CW4gF7v+k5SAGNt3hNE7oPhnyGGcT 6xjsVvD6zyG6VVPPUuza9/ZUyJ/ob2ZksRMDlRE3yG9SCS8yWOPCgw9Qzk81tDLswOJRI4 VMzTbbHYTheG6sfOKehopNGJkIXe84M= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-259-mzphOIU0Nq-eZ7zRHLmc0w-1; Thu, 09 Jan 2020 09:57:59 -0500 X-MC-Unique: mzphOIU0Nq-eZ7zRHLmc0w-1 Received: by mail-qt1-f198.google.com with SMTP id 38so4330781qty.15 for ; Thu, 09 Jan 2020 06:57:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4C6ZGVemG6J37mDpNjuVXzVMZmBhm9r1bGAW/2NgwDc=; b=oUUSiQNQ/CL1+TVvUVst6pgUK+KYX2rq+ClQzrTgFbxAeKBSYChoF0PbgD9H4/WGZz 4XW9WNR/ybgw16D6jHTPygZU7A2a0SqYmrOe2pPJ1Zk5/1QOwklgKCwFKrznylfX14Tn Ts1kr0pQf0BTA/3bGPfx5uFqe+a6BpMFQkzeo+X//B9wDddu7ei7+xRbJmRxemELxCxw /LQqwyaK90oeyuNlZMlTBtvDCPsJU/MEH/kweqqHqI7DaiJDEaTVAzCoA7B/BqZe0zvm SGRpj5TCAvn3mUq+DOhZs8kFD15xtJ9Aryvh1Rll6B13iR4oU+AxqYloqTFnfNVzsunx lomA== X-Gm-Message-State: APjAAAXtuCwdvkE4109/BjKFqUwIxVjSgRstkYnY/BJUBdnNz51WNLqu twVWxK+y4XrulFUr0hOc3iaqbSw1o86fBaZanqI4WAKwCTJe0TMerTMZ/5Fwp6CRG1vQItif1fg 1YUPtItyZYPlS X-Received: by 2002:aed:3f32:: with SMTP id p47mr8522008qtf.374.1578581878243; Thu, 09 Jan 2020 06:57:58 -0800 (PST) X-Google-Smtp-Source: APXvYqzlM1ry2WKKrMYHxQEBp9Y5kFIrXOvo1VCZxj91TgHWTVd0cvi099C4YRjI1OhvTkYkH7/USw== X-Received: by 2002:aed:3f32:: with SMTP id p47mr8521977qtf.374.1578581877922; Thu, 09 Jan 2020 06:57:57 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.57.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:57:56 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 09/21] KVM: X86: Don't track dirty for KVM_SET_[TSS_ADDR|IDENTITY_MAP_ADDR] Date: Thu, 9 Jan 2020 09:57:17 -0500 Message-Id: <20200109145729.32898-10-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Originally, we have three code paths that can dirty a page without vcpu context for X86: - init_rmode_identity_map - init_rmode_tss - kvmgt_rw_gpa init_rmode_identity_map and init_rmode_tss will be setup on destination VM no matter what (and the guest cannot even see them), so it does not make sense to track them at all. To do this, allow __x86_set_memory_region() to return the userspace address that just allocated to the caller. Then in both of the functions we directly write to the userspace address instead of calling kvm_write_*() APIs. We need to make sure that we have the slots_lock held when accessing the userspace address. Another trivial change is that we don't need to explicitly clear the identity page table root in init_rmode_identity_map() because no matter what we'll write to the whole page with 4M huge page entries. Suggested-by: Paolo Bonzini Signed-off-by: Peter Xu --- arch/x86/include/asm/kvm_host.h | 3 +- arch/x86/kvm/svm.c | 3 +- arch/x86/kvm/vmx/vmx.c | 68 ++++++++++++++++----------------- arch/x86/kvm/x86.c | 18 +++++++-- 4 files changed, 51 insertions(+), 41 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index eb6673c7d2e3..f536d139b3d2 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1618,7 +1618,8 @@ void __kvm_request_immediate_exit(struct kvm_vcpu *vcpu); int kvm_is_in_guest(void); -int __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size); +int __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size, + unsigned long *uaddr); bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu); bool kvm_vcpu_is_bsp(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index 8f1b715dfde8..03a344ce7b66 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -1698,7 +1698,8 @@ static int avic_init_access_page(struct kvm_vcpu *vcpu) ret = __x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT, APIC_DEFAULT_PHYS_BASE, - PAGE_SIZE); + PAGE_SIZE, + NULL); if (ret) goto out; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 7e3d370209e0..62175a246bcc 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -3441,34 +3441,28 @@ static bool guest_state_valid(struct kvm_vcpu *vcpu) return true; } -static int init_rmode_tss(struct kvm *kvm) +static int init_rmode_tss(struct kvm *kvm, unsigned long *uaddr) { - gfn_t fn; + const void *zero_page = (const void *) __va(page_to_phys(ZERO_PAGE(0))); u16 data = 0; int idx, r; - idx = srcu_read_lock(&kvm->srcu); - fn = to_kvm_vmx(kvm)->tss_addr >> PAGE_SHIFT; - r = kvm_clear_guest_page(kvm, fn, 0, PAGE_SIZE); - if (r < 0) - goto out; + for (idx = 0; idx < 3; idx++) { + r = __copy_to_user((void __user *)uaddr + PAGE_SIZE * idx, + zero_page, PAGE_SIZE); + if (r) + return -EFAULT; + } + data = TSS_BASE_SIZE + TSS_REDIRECTION_SIZE; - r = kvm_write_guest_page(kvm, fn++, &data, - TSS_IOPB_BASE_OFFSET, sizeof(u16)); - if (r < 0) - goto out; - r = kvm_clear_guest_page(kvm, fn++, 0, PAGE_SIZE); - if (r < 0) - goto out; - r = kvm_clear_guest_page(kvm, fn, 0, PAGE_SIZE); - if (r < 0) - goto out; + r = __copy_to_user((void __user *)uaddr + TSS_IOPB_BASE_OFFSET, + &data, sizeof(data)); + if (r) + return -EFAULT; + data = ~0; - r = kvm_write_guest_page(kvm, fn, &data, - RMODE_TSS_SIZE - 2 * PAGE_SIZE - 1, - sizeof(u8)); -out: - srcu_read_unlock(&kvm->srcu, idx); + r = __copy_to_user((void __user *)uaddr - 1, &data, sizeof(data)); + return r; } @@ -3478,6 +3472,7 @@ static int init_rmode_identity_map(struct kvm *kvm) int i, r = 0; kvm_pfn_t identity_map_pfn; u32 tmp; + unsigned long *uaddr = NULL; /* Protect kvm_vmx->ept_identity_pagetable_done. */ mutex_lock(&kvm->slots_lock); @@ -3490,21 +3485,21 @@ static int init_rmode_identity_map(struct kvm *kvm) identity_map_pfn = kvm_vmx->ept_identity_map_addr >> PAGE_SHIFT; r = __x86_set_memory_region(kvm, IDENTITY_PAGETABLE_PRIVATE_MEMSLOT, - kvm_vmx->ept_identity_map_addr, PAGE_SIZE); + kvm_vmx->ept_identity_map_addr, PAGE_SIZE, + uaddr); if (r < 0) goto out; - r = kvm_clear_guest_page(kvm, identity_map_pfn, 0, PAGE_SIZE); - if (r < 0) - goto out; /* Set up identity-mapping pagetable for EPT in real mode */ for (i = 0; i < PT32_ENT_PER_PAGE; i++) { tmp = (i << 22) + (_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | _PAGE_ACCESSED | _PAGE_DIRTY | _PAGE_PSE); - r = kvm_write_guest_page(kvm, identity_map_pfn, - &tmp, i * sizeof(tmp), sizeof(tmp)); - if (r < 0) + r = __copy_to_user((void __user *)uaddr + i * sizeof(tmp), + &tmp, sizeof(tmp)); + if (r) { + r = -EFAULT; goto out; + } } kvm_vmx->ept_identity_pagetable_done = true; @@ -3537,7 +3532,7 @@ static int alloc_apic_access_page(struct kvm *kvm) if (kvm->arch.apic_access_page_done) goto out; r = __x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT, - APIC_DEFAULT_PHYS_BASE, PAGE_SIZE); + APIC_DEFAULT_PHYS_BASE, PAGE_SIZE, NULL); if (r) goto out; @@ -4478,19 +4473,22 @@ static int vmx_interrupt_allowed(struct kvm_vcpu *vcpu) static int vmx_set_tss_addr(struct kvm *kvm, unsigned int addr) { int ret; + unsigned long *uaddr = NULL; if (enable_unrestricted_guest) return 0; mutex_lock(&kvm->slots_lock); ret = __x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, addr, - PAGE_SIZE * 3); - mutex_unlock(&kvm->slots_lock); - + PAGE_SIZE * 3, uaddr); if (ret) - return ret; + goto out; + to_kvm_vmx(kvm)->tss_addr = addr; - return init_rmode_tss(kvm); + ret = init_rmode_tss(kvm, uaddr); +out: + mutex_unlock(&kvm->slots_lock); + return ret; } static int vmx_set_identity_map_addr(struct kvm *kvm, u64 ident_addr) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c4d3972dcd14..ff97782b3919 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9584,7 +9584,15 @@ void kvm_arch_sync_events(struct kvm *kvm) kvm_free_pit(kvm); } -int __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size) +/* + * If `uaddr' is specified, `*uaddr' will be returned with the + * userspace address that was just allocated. `uaddr' is only + * meaningful if the function returns zero, and `uaddr' will only be + * valid when with either the slots_lock or with the SRCU read lock + * held. After we release the lock, the returned `uaddr' will be invalid. + */ +int __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size, + unsigned long *uaddr) { int i, r; unsigned long hva; @@ -9608,6 +9616,8 @@ int __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size) MAP_SHARED | MAP_ANONYMOUS, 0); if (IS_ERR((void *)hva)) return PTR_ERR((void *)hva); + if (uaddr) + *uaddr = hva; } else { if (!slot->npages) return 0; @@ -9651,10 +9661,10 @@ void kvm_arch_destroy_vm(struct kvm *kvm) */ mutex_lock(&kvm->slots_lock); __x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT, - 0, 0); + 0, 0, NULL); __x86_set_memory_region(kvm, IDENTITY_PAGETABLE_PRIVATE_MEMSLOT, - 0, 0); - __x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, 0, 0); + 0, 0, NULL); + __x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, 0, 0, NULL); mutex_unlock(&kvm->slots_lock); } if (kvm_x86_ops->vm_destroy) From patchwork Thu Jan 9 14:57:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325917 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F1EED109A for ; Thu, 9 Jan 2020 14:59:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D09D92072A for ; Thu, 9 Jan 2020 14:59:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KcjJb8fi" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732028AbgAIO6L (ORCPT ); Thu, 9 Jan 2020 09:58:11 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:41140 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1731992AbgAIO6D (ORCPT ); Thu, 9 Jan 2020 09:58:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581882; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KKTi945t71agkV8/WyMDLT+ZjNMTeinowRyga3iDYLQ=; b=KcjJb8fivwjey1PYW1Mp+Porw5z2suwodU5Gildx+BD2G1mEHbh2CDDplLIR4yc6Q4ty3Y H1K9jyBBEKNIdPKRXYlhXdi501jViO0NaREgicO+5JQlTTYWktmQHTRpY/U+PwSHOpEWxp EifDC35z9GhdfgYZLxpN+RODz9EWZpM= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-78-ujAlL0IFNiGdOOYuUMN-_Q-1; Thu, 09 Jan 2020 09:58:01 -0500 X-MC-Unique: ujAlL0IFNiGdOOYuUMN-_Q-1 Received: by mail-qk1-f200.google.com with SMTP id w64so4326076qka.3 for ; Thu, 09 Jan 2020 06:58:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KKTi945t71agkV8/WyMDLT+ZjNMTeinowRyga3iDYLQ=; b=sdlXm0t/rgOM8/4HflPBlhR+4WU/iH93QolLX/vV2MMArVfafuJXNHKz8hHAGvhdPw IcKsuChbl/7UK8FpBSNCqNgMnHqfRUomI0cxeKIAYInle2Glw+j3/frNy7gvA00cxfpm bcc3nlbUhKKxoWI1/9U9E3ox7IYnutEG3UGxxL/XK3nVD/pjxUt/bCGkS6qzH0sl3d3G yggFFS7oMmdU1keQSHJWk/w7TC3lM9ISvMu8bjV4YsmJqxAYRfwtLjm8VZhGSCgjudWO NpbI6LU4kVkr3OSNxKa9S7zs0v0GCGKBISllD6T9JypfG9cbIq6uZ3s6worqDhYrjILm eACA== X-Gm-Message-State: APjAAAVG2QO+LzNXcAaIPH/yux+RBiO7hCMWHW3Q5RvnidMGOk4j5dOe F3JmA5qGo8F6h1wUAO6Daw8ZFQYOzr+B4XNmdbpqAvkBxlHhprPkFjh9CdiVfXOC5ghhglUt11o ZASiwmJ0MOHW4 X-Received: by 2002:aed:3e83:: with SMTP id n3mr8255313qtf.322.1578581880517; Thu, 09 Jan 2020 06:58:00 -0800 (PST) X-Google-Smtp-Source: APXvYqzm+32rbTx5mDTUHNQfE3lH6k5rH+Ap/p2Tv6k1bif/l76ucLLtNQj90U1GMr8JgPQrYwqJbA== X-Received: by 2002:aed:3e83:: with SMTP id n3mr8255271qtf.322.1578581880060; Thu, 09 Jan 2020 06:58:00 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.57.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:57:58 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 10/21] KVM: Pass in kvm pointer into mark_page_dirty_in_slot() Date: Thu, 9 Jan 2020 09:57:18 -0500 Message-Id: <20200109145729.32898-11-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The context will be needed to implement the kvm dirty ring. Reviewed-by: Paolo Bonzini Signed-off-by: Peter Xu --- virt/kvm/kvm_main.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 1fd204f27028..028dfc27479b 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -144,7 +144,9 @@ static void hardware_disable_all(void); static void kvm_io_bus_destroy(struct kvm_io_bus *bus); -static void mark_page_dirty_in_slot(struct kvm_memory_slot *memslot, gfn_t gfn); +static void mark_page_dirty_in_slot(struct kvm *kvm, + struct kvm_memory_slot *memslot, + gfn_t gfn); __visible bool kvm_rebooting; EXPORT_SYMBOL_GPL(kvm_rebooting); @@ -2062,7 +2064,8 @@ int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu, gpa_t gpa, } EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_atomic); -static int __kvm_write_guest_page(struct kvm_memory_slot *memslot, gfn_t gfn, +static int __kvm_write_guest_page(struct kvm *kvm, + struct kvm_memory_slot *memslot, gfn_t gfn, const void *data, int offset, int len) { int r; @@ -2074,7 +2077,7 @@ static int __kvm_write_guest_page(struct kvm_memory_slot *memslot, gfn_t gfn, r = __copy_to_user((void __user *)addr + offset, data, len); if (r) return -EFAULT; - mark_page_dirty_in_slot(memslot, gfn); + mark_page_dirty_in_slot(kvm, memslot, gfn); return 0; } @@ -2083,7 +2086,7 @@ int kvm_write_guest_page(struct kvm *kvm, gfn_t gfn, { struct kvm_memory_slot *slot = gfn_to_memslot(kvm, gfn); - return __kvm_write_guest_page(slot, gfn, data, offset, len); + return __kvm_write_guest_page(kvm, slot, gfn, data, offset, len); } EXPORT_SYMBOL_GPL(kvm_write_guest_page); @@ -2092,7 +2095,7 @@ int kvm_vcpu_write_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, { struct kvm_memory_slot *slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn); - return __kvm_write_guest_page(slot, gfn, data, offset, len); + return __kvm_write_guest_page(vcpu->kvm, slot, gfn, data, offset, len); } EXPORT_SYMBOL_GPL(kvm_vcpu_write_guest_page); @@ -2206,7 +2209,7 @@ int kvm_write_guest_offset_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, r = __copy_to_user((void __user *)ghc->hva + offset, data, len); if (r) return -EFAULT; - mark_page_dirty_in_slot(ghc->memslot, gpa >> PAGE_SHIFT); + mark_page_dirty_in_slot(kvm, ghc->memslot, gpa >> PAGE_SHIFT); return 0; } @@ -2271,7 +2274,8 @@ int kvm_clear_guest(struct kvm *kvm, gpa_t gpa, unsigned long len) } EXPORT_SYMBOL_GPL(kvm_clear_guest); -static void mark_page_dirty_in_slot(struct kvm_memory_slot *memslot, +static void mark_page_dirty_in_slot(struct kvm *kvm, + struct kvm_memory_slot *memslot, gfn_t gfn) { if (memslot && memslot->dirty_bitmap) { @@ -2286,7 +2290,7 @@ void mark_page_dirty(struct kvm *kvm, gfn_t gfn) struct kvm_memory_slot *memslot; memslot = gfn_to_memslot(kvm, gfn); - mark_page_dirty_in_slot(memslot, gfn); + mark_page_dirty_in_slot(kvm, memslot, gfn); } EXPORT_SYMBOL_GPL(mark_page_dirty); @@ -2295,7 +2299,7 @@ void kvm_vcpu_mark_page_dirty(struct kvm_vcpu *vcpu, gfn_t gfn) struct kvm_memory_slot *memslot; memslot = kvm_vcpu_gfn_to_memslot(vcpu, gfn); - mark_page_dirty_in_slot(memslot, gfn); + mark_page_dirty_in_slot(vcpu->kvm, memslot, gfn); } EXPORT_SYMBOL_GPL(kvm_vcpu_mark_page_dirty); From patchwork Thu Jan 9 14:57:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325919 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4F23D109A for ; Thu, 9 Jan 2020 14:59:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 19EC22072E for ; Thu, 9 Jan 2020 14:59:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AfXEHq0c" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732009AbgAIO7T (ORCPT ); Thu, 9 Jan 2020 09:59:19 -0500 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:40371 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732005AbgAIO6H (ORCPT ); Thu, 9 Jan 2020 09:58:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581885; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GjELQ4ch6bUfpWMojlZkE0ODIq70oi5Kwek7NgAA0/Y=; b=AfXEHq0c5ejjCgOQ9pK23hC3DwjwyDBXZZfXALckWelocuxX2TOu8JkLgAw2jQpMagnzAp xdNbwF8G52D8OryprlkQufJAtl30oc3TIL4aGs37cvf/PYVqsFe7ndkKGbPizcmC0Uapd9 GhJbS+zoKV3/IoyrkleYUPwudHbytgs= Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-262-lJbKYCYXNySNP5JPvIgHQw-1; Thu, 09 Jan 2020 09:58:04 -0500 X-MC-Unique: lJbKYCYXNySNP5JPvIgHQw-1 Received: by mail-qk1-f198.google.com with SMTP id m13so4283430qka.9 for ; Thu, 09 Jan 2020 06:58:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GjELQ4ch6bUfpWMojlZkE0ODIq70oi5Kwek7NgAA0/Y=; b=cF1/330Q9y96uQRQFpJIUzWgZ/i1HQ3OxUR2CD1YCSlCWkfQ3rT4qUUq6ReXZ08wVV rVE7Nts2qz6Wo9t8jbz7wvd+TM5X8k9wk6TLma6SUZ8S4VwTj9Vv7xyvMRelhtSCaHxR 5YeLt47JIaddT77lCvPQ5De3wCE5bjRaqzCq7aObInnwEfYrxKHV4A6CwJDxLq7Eorxx WsRogeVpMLpgTlmFyZh5IOs6y4a0hge0yL/9XVf61u5QKy487e0GLzPg51/ulOLFob+9 YfPaHFadpxAOuwu7Xbp4OoE8HsqfrJLrVCVTu4vIAs69W5SPZAtw0mquVKeruYYQTwVQ tTiA== X-Gm-Message-State: APjAAAXJHZOHtVFhpPVL1qSOuzU/Lt4ZFNtq6fYQKI/Ll/82WWKVRkZ4 lMpt7zdL7Dx7kuoLJiKl4ijqa/tKUV7PbK20d7BikR/X2BLuwLfyPlAm2YWM3bR6PUcPp74gX2R 4dNEzAl8XoflH X-Received: by 2002:a05:620a:6cc:: with SMTP id 12mr10223277qky.307.1578581883431; Thu, 09 Jan 2020 06:58:03 -0800 (PST) X-Google-Smtp-Source: APXvYqzP44DUKoC71rp7Hfr+zAsn6v+OupR+yTVgvz266MrANVKdmmxph98eVT1lNqyPZF1oF3il/A== X-Received: by 2002:a05:620a:6cc:: with SMTP id 12mr10223249qky.307.1578581883060; Thu, 09 Jan 2020 06:58:03 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.58.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:58:00 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 11/21] KVM: Move running VCPU from ARM to common code Date: Thu, 9 Jan 2020 09:57:19 -0500 Message-Id: <20200109145729.32898-12-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Paolo Bonzini For ring-based dirty log tracking, it will be more efficient to account writes during schedule-out or schedule-in to the currently running VCPU. We would like to do it even if the write doesn't use the current VCPU's address space, as is the case for cached writes (see commit 4e335d9e7ddb, "Revert "KVM: Support vCPU-based gfn->hva cache"", 2017-05-02). Therefore, add a mechanism to track the currently-loaded kvm_vcpu struct. There is already something similar in KVM/ARM; one important difference is that kvm_arch_vcpu_{load,put} have two callers in virt/kvm/kvm_main.c: we have to update both the architecture-independent vcpu_{load,put} and the preempt notifiers. Another change made in the process is to allow using kvm_get_running_vcpu() in preemptible code. This is allowed because preempt notifiers ensure that the value does not change even after the VCPU thread is migrated. Signed-off-by: Paolo Bonzini Reviewed-by: Paolo Bonzini Signed-off-by: Peter Xu --- arch/arm/include/asm/kvm_host.h | 2 -- arch/arm64/include/asm/kvm_host.h | 2 -- include/linux/kvm_host.h | 3 +++ virt/kvm/arm/arch_timer.c | 2 +- virt/kvm/arm/arm.c | 29 ----------------------------- virt/kvm/arm/perf.c | 6 +++--- virt/kvm/arm/vgic/vgic-mmio.c | 15 +++------------ virt/kvm/kvm_main.c | 25 ++++++++++++++++++++++++- 8 files changed, 34 insertions(+), 50 deletions(-) diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h index 556cd818eccf..abc3f6f3ad76 100644 --- a/arch/arm/include/asm/kvm_host.h +++ b/arch/arm/include/asm/kvm_host.h @@ -284,8 +284,6 @@ int kvm_arm_copy_reg_indices(struct kvm_vcpu *vcpu, u64 __user *indices); int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end); int kvm_test_age_hva(struct kvm *kvm, unsigned long hva); -struct kvm_vcpu *kvm_arm_get_running_vcpu(void); -struct kvm_vcpu __percpu **kvm_get_running_vcpus(void); void kvm_arm_halt_guest(struct kvm *kvm); void kvm_arm_resume_guest(struct kvm *kvm); diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index c61260cf63c5..12302f9035f9 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -446,8 +446,6 @@ int kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte); int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end); int kvm_test_age_hva(struct kvm *kvm, unsigned long hva); -struct kvm_vcpu *kvm_arm_get_running_vcpu(void); -struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void); void kvm_arm_halt_guest(struct kvm *kvm); void kvm_arm_resume_guest(struct kvm *kvm); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 763adf8c47b0..cbd633ece959 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1340,6 +1340,9 @@ static inline void kvm_vcpu_set_dy_eligible(struct kvm_vcpu *vcpu, bool val) } #endif /* CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT */ +struct kvm_vcpu *kvm_get_running_vcpu(void); +struct kvm_vcpu __percpu **kvm_get_running_vcpus(void); + #ifdef CONFIG_HAVE_KVM_IRQ_BYPASS bool kvm_arch_has_irq_bypass(void); int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *, diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c index f182b2380345..63dd6f27997c 100644 --- a/virt/kvm/arm/arch_timer.c +++ b/virt/kvm/arm/arch_timer.c @@ -1022,7 +1022,7 @@ static bool timer_irqs_are_valid(struct kvm_vcpu *vcpu) bool kvm_arch_timer_get_input_level(int vintid) { - struct kvm_vcpu *vcpu = kvm_arm_get_running_vcpu(); + struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); struct arch_timer_context *timer; if (vintid == vcpu_vtimer(vcpu)->irq.irq) diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c index 8de4daf25097..b00a9870e5ec 100644 --- a/virt/kvm/arm/arm.c +++ b/virt/kvm/arm/arm.c @@ -51,9 +51,6 @@ __asm__(".arch_extension virt"); DEFINE_PER_CPU(kvm_host_data_t, kvm_host_data); static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); -/* Per-CPU variable containing the currently running vcpu. */ -static DEFINE_PER_CPU(struct kvm_vcpu *, kvm_arm_running_vcpu); - /* The VMID used in the VTTBR */ static atomic64_t kvm_vmid_gen = ATOMIC64_INIT(1); static u32 kvm_next_vmid; @@ -62,31 +59,8 @@ static DEFINE_SPINLOCK(kvm_vmid_lock); static bool vgic_present; static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled); - -static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu) -{ - __this_cpu_write(kvm_arm_running_vcpu, vcpu); -} - DEFINE_STATIC_KEY_FALSE(userspace_irqchip_in_use); -/** - * kvm_arm_get_running_vcpu - get the vcpu running on the current CPU. - * Must be called from non-preemptible context - */ -struct kvm_vcpu *kvm_arm_get_running_vcpu(void) -{ - return __this_cpu_read(kvm_arm_running_vcpu); -} - -/** - * kvm_arm_get_running_vcpus - get the per-CPU array of currently running vcpus. - */ -struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void) -{ - return &kvm_arm_running_vcpu; -} - int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE; @@ -406,7 +380,6 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) vcpu->cpu = cpu; vcpu->arch.host_cpu_context = &cpu_data->host_ctxt; - kvm_arm_set_running_vcpu(vcpu); kvm_vgic_load(vcpu); kvm_timer_vcpu_load(vcpu); kvm_vcpu_load_sysregs(vcpu); @@ -432,8 +405,6 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) kvm_vcpu_pmu_restore_host(vcpu); vcpu->cpu = -1; - - kvm_arm_set_running_vcpu(NULL); } static void vcpu_power_off(struct kvm_vcpu *vcpu) diff --git a/virt/kvm/arm/perf.c b/virt/kvm/arm/perf.c index 918cdc3839ea..d45b8b9a4415 100644 --- a/virt/kvm/arm/perf.c +++ b/virt/kvm/arm/perf.c @@ -13,14 +13,14 @@ static int kvm_is_in_guest(void) { - return kvm_arm_get_running_vcpu() != NULL; + return kvm_get_running_vcpu() != NULL; } static int kvm_is_user_mode(void) { struct kvm_vcpu *vcpu; - vcpu = kvm_arm_get_running_vcpu(); + vcpu = kvm_get_running_vcpu(); if (vcpu) return !vcpu_mode_priv(vcpu); @@ -32,7 +32,7 @@ static unsigned long kvm_get_guest_ip(void) { struct kvm_vcpu *vcpu; - vcpu = kvm_arm_get_running_vcpu(); + vcpu = kvm_get_running_vcpu(); if (vcpu) return *vcpu_pc(vcpu); diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c index 0d090482720d..d656ebd5f9d4 100644 --- a/virt/kvm/arm/vgic/vgic-mmio.c +++ b/virt/kvm/arm/vgic/vgic-mmio.c @@ -190,15 +190,6 @@ unsigned long vgic_mmio_read_pending(struct kvm_vcpu *vcpu, * value later will give us the same value as we update the per-CPU variable * in the preempt notifier handlers. */ -static struct kvm_vcpu *vgic_get_mmio_requester_vcpu(void) -{ - struct kvm_vcpu *vcpu; - - preempt_disable(); - vcpu = kvm_arm_get_running_vcpu(); - preempt_enable(); - return vcpu; -} /* Must be called with irq->irq_lock held */ static void vgic_hw_irq_spending(struct kvm_vcpu *vcpu, struct vgic_irq *irq, @@ -221,7 +212,7 @@ void vgic_mmio_write_spending(struct kvm_vcpu *vcpu, gpa_t addr, unsigned int len, unsigned long val) { - bool is_uaccess = !vgic_get_mmio_requester_vcpu(); + bool is_uaccess = !kvm_get_running_vcpu(); u32 intid = VGIC_ADDR_TO_INTID(addr, 1); int i; unsigned long flags; @@ -274,7 +265,7 @@ void vgic_mmio_write_cpending(struct kvm_vcpu *vcpu, gpa_t addr, unsigned int len, unsigned long val) { - bool is_uaccess = !vgic_get_mmio_requester_vcpu(); + bool is_uaccess = !kvm_get_running_vcpu(); u32 intid = VGIC_ADDR_TO_INTID(addr, 1); int i; unsigned long flags; @@ -335,7 +326,7 @@ static void vgic_mmio_change_active(struct kvm_vcpu *vcpu, struct vgic_irq *irq, bool active) { unsigned long flags; - struct kvm_vcpu *requester_vcpu = vgic_get_mmio_requester_vcpu(); + struct kvm_vcpu *requester_vcpu = kvm_get_running_vcpu(); raw_spin_lock_irqsave(&irq->irq_lock, flags); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 028dfc27479b..5bbd8b8730fa 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -108,6 +108,7 @@ struct kmem_cache *kvm_vcpu_cache; EXPORT_SYMBOL_GPL(kvm_vcpu_cache); static __read_mostly struct preempt_ops kvm_preempt_ops; +static DEFINE_PER_CPU(struct kvm_vcpu *, kvm_running_vcpu); struct dentry *kvm_debugfs_dir; EXPORT_SYMBOL_GPL(kvm_debugfs_dir); @@ -199,6 +200,8 @@ bool kvm_is_reserved_pfn(kvm_pfn_t pfn) void vcpu_load(struct kvm_vcpu *vcpu) { int cpu = get_cpu(); + + __this_cpu_write(kvm_running_vcpu, vcpu); preempt_notifier_register(&vcpu->preempt_notifier); kvm_arch_vcpu_load(vcpu, cpu); put_cpu(); @@ -210,6 +213,7 @@ void vcpu_put(struct kvm_vcpu *vcpu) preempt_disable(); kvm_arch_vcpu_put(vcpu); preempt_notifier_unregister(&vcpu->preempt_notifier); + __this_cpu_write(kvm_running_vcpu, NULL); preempt_enable(); } EXPORT_SYMBOL_GPL(vcpu_put); @@ -4297,8 +4301,8 @@ static void kvm_sched_in(struct preempt_notifier *pn, int cpu) WRITE_ONCE(vcpu->preempted, false); WRITE_ONCE(vcpu->ready, false); + __this_cpu_write(kvm_running_vcpu, vcpu); kvm_arch_sched_in(vcpu, cpu); - kvm_arch_vcpu_load(vcpu, cpu); } @@ -4312,6 +4316,25 @@ static void kvm_sched_out(struct preempt_notifier *pn, WRITE_ONCE(vcpu->ready, true); } kvm_arch_vcpu_put(vcpu); + __this_cpu_write(kvm_running_vcpu, NULL); +} + +/** + * kvm_get_running_vcpu - get the vcpu running on the current CPU. + * Thanks to preempt notifiers, this can also be called from + * preemptible context. + */ +struct kvm_vcpu *kvm_get_running_vcpu(void) +{ + return __this_cpu_read(kvm_running_vcpu); +} + +/** + * kvm_get_running_vcpus - get the per-CPU array of currently running vcpus. + */ +struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void) +{ + return &kvm_running_vcpu; } static void check_processor_compat(void *rtn) From patchwork Thu Jan 9 14:57:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325899 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 06550109A for ; Thu, 9 Jan 2020 14:58:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B9F2D2072A for ; Thu, 9 Jan 2020 14:58:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DWcz1pe8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730377AbgAIO6U (ORCPT ); Thu, 9 Jan 2020 09:58:20 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:60461 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732054AbgAIO6T (ORCPT ); Thu, 9 Jan 2020 09:58:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581895; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XckEDLRENN4NMzH+7zRGAj1B6DfQa06b5jIzfBoE4SY=; b=DWcz1pe80k4WNQ14tjTzFgVudp9r6ntzf6rifXxzF51pvzugCNVbUy38wynzd5CwtRoQy8 KsmmrGG+/IGQvpGO7xhhWgoFmValLKBMAFCOSelaM9OKkyIojWKD9IRi6IuUfgcFIX71kl KBsE/2VkPKCX6EN0KgtGOWjpijRI/2s= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-72-f6kL_XCFPOK8DwQnmsZ0gA-1; Thu, 09 Jan 2020 09:58:12 -0500 X-MC-Unique: f6kL_XCFPOK8DwQnmsZ0gA-1 Received: by mail-qk1-f197.google.com with SMTP id x127so4302968qkb.0 for ; Thu, 09 Jan 2020 06:58:12 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XckEDLRENN4NMzH+7zRGAj1B6DfQa06b5jIzfBoE4SY=; b=eDmzNdfMbdxdQPHbHZC3zSP3Ma0jGGSfgmjY1GsKA6eRHSzzjo+NVpoIyRiQID1HNC j0gXM7ASO4eBDpvIggPVPgc6aKBvSTvIVZ4IrceOao+s68XKoULlKP5iNzYgIfkKFm5v rsbPyf7qU94cZgoKdz7x2HAnh2SGcRIPqy6gGABsRrTwPW/950EpPgYXEtymPXbROYq+ 7oEv9kgq0Q+J3S+kzTlCwIa6LBLM2MghJZ4WK/B5Itvq2CI90b9UThFi42ilpwNycPfZ VEkIDEq4v2YkZWMJ8Kr2EZUjZz1mUo8xW5k1D/OrPpEB/+5gzGujnS18dybNypPSS7ec xNmQ== X-Gm-Message-State: APjAAAXdZS6WXIrWQw7LttYscfnLA2EbwrFU02VDCQff5BO7Fk38t94g ntYg7At1Q623Mf0x8OtlMewe5Ppmq7BI+lk8s2sj2PSXAAarZlRrDGKVlgTvU2k4pNt6sR7hxKe xQKxM69LFtPS6 X-Received: by 2002:a37:674a:: with SMTP id b71mr10155542qkc.471.1578581887119; Thu, 09 Jan 2020 06:58:07 -0800 (PST) X-Google-Smtp-Source: APXvYqzHay5DriPGBA5eXIqyNNSP4eRppf98PZKs0Xp4yR2ktb6kAi6i+zD3cScUlnWmUz6DdCdHOA== X-Received: by 2002:a37:674a:: with SMTP id b71mr10155477qkc.471.1578581886251; Thu, 09 Jan 2020 06:58:06 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.58.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:58:04 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" , Lei Cao Subject: [PATCH v3 12/21] KVM: X86: Implement ring-based dirty memory tracking Date: Thu, 9 Jan 2020 09:57:20 -0500 Message-Id: <20200109145729.32898-13-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This patch is heavily based on previous work from Lei Cao and Paolo Bonzini . [1] KVM currently uses large bitmaps to track dirty memory. These bitmaps are copied to userspace when userspace queries KVM for its dirty page information. The use of bitmaps is mostly sufficient for live migration, as large parts of memory are be dirtied from one log-dirty pass to another. However, in a checkpointing system, the number of dirty pages is small and in fact it is often bounded---the VM is paused when it has dirtied a pre-defined number of pages. Traversing a large, sparsely populated bitmap to find set bits is time-consuming, as is copying the bitmap to user-space. A similar issue will be there for live migration when the guest memory is huge while the page dirty procedure is trivial. In that case for each dirty sync we need to pull the whole dirty bitmap to userspace and analyse every bit even if it's mostly zeros. The preferred data structure for above scenarios is a dense list of guest frame numbers (GFN). This patch series stores the dirty list in kernel memory that can be memory mapped into userspace to allow speedy harvesting. This patch enables dirty ring for X86 only. However it should be easily extended to other archs as well. [1] https://patchwork.kernel.org/patch/10471409/ Signed-off-by: Lei Cao Signed-off-by: Paolo Bonzini Signed-off-by: Peter Xu Reported-by: kbuild test robot Reported-by: kbuild test robot --- Documentation/virt/kvm/api.txt | 89 ++++++++++++++++++ arch/x86/include/asm/kvm_host.h | 3 + arch/x86/include/uapi/asm/kvm.h | 1 + arch/x86/kvm/Makefile | 3 +- arch/x86/kvm/mmu/mmu.c | 6 ++ arch/x86/kvm/vmx/vmx.c | 7 ++ arch/x86/kvm/x86.c | 9 ++ include/linux/kvm_dirty_ring.h | 55 +++++++++++ include/linux/kvm_host.h | 26 +++++ include/trace/events/kvm.h | 78 +++++++++++++++ include/uapi/linux/kvm.h | 33 +++++++ virt/kvm/dirty_ring.c | 162 ++++++++++++++++++++++++++++++++ virt/kvm/kvm_main.c | 137 ++++++++++++++++++++++++++- 13 files changed, 606 insertions(+), 3 deletions(-) create mode 100644 include/linux/kvm_dirty_ring.h create mode 100644 virt/kvm/dirty_ring.c diff --git a/Documentation/virt/kvm/api.txt b/Documentation/virt/kvm/api.txt index ebb37b34dcfc..708c3e0f7eae 100644 --- a/Documentation/virt/kvm/api.txt +++ b/Documentation/virt/kvm/api.txt @@ -231,6 +231,7 @@ Based on their initialization different VMs may have different capabilities. It is thus encouraged to use the vm ioctl to query for capabilities (available with KVM_CAP_CHECK_EXTENSION_VM on the vm fd) + 4.5 KVM_GET_VCPU_MMAP_SIZE Capability: basic @@ -243,6 +244,18 @@ The KVM_RUN ioctl (cf.) communicates with userspace via a shared memory region. This ioctl returns the size of that region. See the KVM_RUN documentation for details. +Besides the size of the KVM_RUN communication region, other areas of +the VCPU file descriptor can be mmap-ed, including: + +- if KVM_CAP_COALESCED_MMIO is available, a page at + KVM_COALESCED_MMIO_PAGE_OFFSET * PAGE_SIZE; for historical reasons, + this page is included in the result of KVM_GET_VCPU_MMAP_SIZE. + KVM_CAP_COALESCED_MMIO is not documented yet. + +- if KVM_CAP_DIRTY_LOG_RING is available, a number of pages at + KVM_DIRTY_LOG_PAGE_OFFSET * PAGE_SIZE. For more information on + KVM_CAP_DIRTY_LOG_RING, see section 8.3. + 4.6 KVM_SET_MEMORY_REGION @@ -5376,6 +5389,7 @@ CPU when the exception is taken. If this virtual SError is taken to EL1 using AArch64, this value will be reported in the ISS field of ESR_ELx. See KVM_CAP_VCPU_EVENTS for more details. + 8.20 KVM_CAP_HYPERV_SEND_IPI Architectures: x86 @@ -5383,6 +5397,7 @@ Architectures: x86 This capability indicates that KVM supports paravirtualized Hyper-V IPI send hypercalls: HvCallSendSyntheticClusterIpi, HvCallSendSyntheticClusterIpiEx. + 8.21 KVM_CAP_HYPERV_DIRECT_TLBFLUSH Architecture: x86 @@ -5396,3 +5411,77 @@ handling by KVM (as some KVM hypercall may be mistakenly treated as TLB flush hypercalls by Hyper-V) so userspace should disable KVM identification in CPUID and only exposes Hyper-V identification. In this case, guest thinks it's running on Hyper-V and only use Hyper-V hypercalls. + +8.22 KVM_CAP_DIRTY_LOG_RING + +Architectures: x86 +Parameters: args[0] - size of the dirty log ring + +KVM is capable of tracking dirty memory using ring buffers that are +mmaped into userspace; there is one dirty ring per vcpu. + +One dirty ring is defined as below internally: + +struct kvm_dirty_ring { + u32 dirty_index; + u32 reset_index; + u32 size; + u32 soft_limit; + struct kvm_dirty_gfn *dirty_gfns; + struct kvm_dirty_ring_indices *indices; + int index; +}; + +Dirty GFNs (Guest Frame Numbers) are stored in the dirty_gfns array. +For each of the dirty entry it's defined as: + +struct kvm_dirty_gfn { + __u32 pad; + __u32 slot; /* as_id | slot_id */ + __u64 offset; +}; + +Most of the ring structure is used by KVM internally, while only the +indices are exposed to userspace: + +struct kvm_dirty_ring_indices { + __u32 avail_index; /* set by kernel */ + __u32 fetch_index; /* set by userspace */ +}; + +The two indices in the ring buffer are free running counters. + +Userspace calls KVM_ENABLE_CAP ioctl right after KVM_CREATE_VM ioctl +to enable this capability for the new guest and set the size of the +rings. It is only allowed before creating any vCPU, and the size of +the ring must be a power of two. The larger the ring buffer, the less +likely the ring is full and the VM is forced to exit to userspace. The +optimal size depends on the workload, but it is recommended that it be +at least 64 KiB (4096 entries). + +Just like for dirty page bitmaps, the buffer tracks writes to +all user memory regions for which the KVM_MEM_LOG_DIRTY_PAGES flag was +set in KVM_SET_USER_MEMORY_REGION. Once a memory region is registered +with the flag set, userspace can start harvesting dirty pages from the +ring buffer. + +To harvest the dirty pages, userspace accesses the mmaped ring buffer +to read the dirty GFNs up to avail_index, and sets the fetch_index +accordingly. This can be done when the guest is running or paused, +and dirty pages need not be collected all at once. After processing +one or more entries in the ring buffer, userspace calls the VM ioctl +KVM_RESET_DIRTY_RINGS to notify the kernel that it has updated +fetch_index and to mark those pages clean. Therefore, the ioctl +must be called *before* reading the content of the dirty pages. + +However, there is a major difference comparing to the +KVM_GET_DIRTY_LOG interface in that when reading the dirty ring from +userspace it's still possible that the kernel has not yet flushed the +hardware dirty buffers into the kernel buffer (which was previously +done by the KVM_GET_DIRTY_LOG ioctl). To achieve that, one needs to +kick the vcpu out for a hardware buffer flush (vmexit) to make sure +all the existing dirty gfns are flushed to the dirty rings. + +If one of the ring buffers is full, the guest will exit to userspace +with the exit reason set to KVM_EXIT_DIRTY_LOG_FULL, and the KVM_RUN +ioctl will return to userspace with zero. diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index f536d139b3d2..3fe18402e6a3 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1181,6 +1181,7 @@ struct kvm_x86_ops { struct kvm_memory_slot *slot, gfn_t offset, unsigned long mask); int (*write_log_dirty)(struct kvm_vcpu *vcpu); + int (*cpu_dirty_log_size)(void); /* pmu operations of sub-arch */ const struct kvm_pmu_ops *pmu_ops; @@ -1666,4 +1667,6 @@ static inline int kvm_cpu_get_apicid(int mps_cpu) #define GET_SMSTATE(type, buf, offset) \ (*(type *)((buf) + (offset) - 0x7e00)) +int kvm_cpu_dirty_log_size(void); + #endif /* _ASM_X86_KVM_HOST_H */ diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 503d3f42da16..b59bf356c478 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -12,6 +12,7 @@ #define KVM_PIO_PAGE_OFFSET 1 #define KVM_COALESCED_MMIO_PAGE_OFFSET 2 +#define KVM_DIRTY_LOG_PAGE_OFFSET 64 #define DE_VECTOR 0 #define DB_VECTOR 1 diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile index b19ef421084d..0acee817adfb 100644 --- a/arch/x86/kvm/Makefile +++ b/arch/x86/kvm/Makefile @@ -5,7 +5,8 @@ ccflags-y += -Iarch/x86/kvm KVM := ../../../virt/kvm kvm-y += $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o \ - $(KVM)/eventfd.o $(KVM)/irqchip.o $(KVM)/vfio.o + $(KVM)/eventfd.o $(KVM)/irqchip.o $(KVM)/vfio.o \ + $(KVM)/dirty_ring.o kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o kvm-y += x86.o emulate.o i8259.o irq.o lapic.o \ diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 7269130ea5e2..621b842a9b7b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -1832,7 +1832,13 @@ int kvm_arch_write_log_dirty(struct kvm_vcpu *vcpu) { if (kvm_x86_ops->write_log_dirty) return kvm_x86_ops->write_log_dirty(vcpu); + return 0; +} +int kvm_cpu_dirty_log_size(void) +{ + if (kvm_x86_ops->cpu_dirty_log_size) + return kvm_x86_ops->cpu_dirty_log_size(); return 0; } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 62175a246bcc..2151de89456d 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7689,6 +7689,7 @@ static __init int hardware_setup(void) kvm_x86_ops->slot_disable_log_dirty = NULL; kvm_x86_ops->flush_log_dirty = NULL; kvm_x86_ops->enable_log_dirty_pt_masked = NULL; + kvm_x86_ops->cpu_dirty_log_size = NULL; } if (!cpu_has_vmx_preemption_timer()) @@ -7753,6 +7754,11 @@ static __exit void hardware_unsetup(void) free_kvm_area(); } +static int vmx_cpu_dirty_log_size(void) +{ + return enable_pml ? PML_ENTITY_NUM : 0; +} + static struct kvm_x86_ops vmx_x86_ops __ro_after_init = { .cpu_has_kvm_support = cpu_has_kvm_support, .disabled_by_bios = vmx_disabled_by_bios, @@ -7875,6 +7881,7 @@ static struct kvm_x86_ops vmx_x86_ops __ro_after_init = { .flush_log_dirty = vmx_flush_log_dirty, .enable_log_dirty_pt_masked = vmx_enable_log_dirty_pt_masked, .write_log_dirty = vmx_write_pml_buffer, + .cpu_dirty_log_size = vmx_cpu_dirty_log_size, .pre_block = vmx_pre_block, .post_block = vmx_post_block, diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ff97782b3919..9c3673592826 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -7998,6 +7998,15 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) bool req_immediate_exit = false; + /* Forbid vmenter if vcpu dirty ring is soft-full */ + if (unlikely(vcpu->kvm->dirty_ring_size && + kvm_dirty_ring_soft_full(&vcpu->dirty_ring))) { + vcpu->run->exit_reason = KVM_EXIT_DIRTY_RING_FULL; + trace_kvm_dirty_ring_exit(vcpu); + r = 0; + goto out; + } + if (kvm_request_pending(vcpu)) { if (kvm_check_request(KVM_REQ_GET_VMCS12_PAGES, vcpu)) { if (unlikely(!kvm_x86_ops->get_vmcs12_pages(vcpu))) { diff --git a/include/linux/kvm_dirty_ring.h b/include/linux/kvm_dirty_ring.h new file mode 100644 index 000000000000..d6fe9e1b7617 --- /dev/null +++ b/include/linux/kvm_dirty_ring.h @@ -0,0 +1,55 @@ +#ifndef KVM_DIRTY_RING_H +#define KVM_DIRTY_RING_H + +/** + * kvm_dirty_ring: KVM internal dirty ring structure + * + * @dirty_index: free running counter that points to the next slot in + * dirty_ring->dirty_gfns, where a new dirty page should go + * @reset_index: free running counter that points to the next dirty page + * in dirty_ring->dirty_gfns for which dirty trap needs to + * be reenabled + * @size: size of the compact list, dirty_ring->dirty_gfns + * @soft_limit: when the number of dirty pages in the list reaches this + * limit, vcpu that owns this ring should exit to userspace + * to allow userspace to harvest all the dirty pages + * @dirty_gfns: the array to keep the dirty gfns + * @indices: the pointer to the @kvm_dirty_ring_indices structure + * of this specific ring + * @index: index of this dirty ring + */ +struct kvm_dirty_ring { + u32 dirty_index; + u32 reset_index; + u32 size; + u32 soft_limit; + struct kvm_dirty_gfn *dirty_gfns; + struct kvm_dirty_ring_indices *indices; + int index; +}; + +u32 kvm_dirty_ring_get_rsvd_entries(void); +int kvm_dirty_ring_alloc(struct kvm_dirty_ring *ring, + struct kvm_dirty_ring_indices *indices, + int index, u32 size); +struct kvm_dirty_ring *kvm_dirty_ring_get(struct kvm *kvm); + +/* + * called with kvm->slots_lock held, returns the number of + * processed pages. + */ +int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring); + +/* + * returns =0: successfully pushed + * <0: unable to push, need to wait + */ +void kvm_dirty_ring_push(struct kvm_dirty_ring *ring, u32 slot, u64 offset); + +/* for use in vm_operations_struct */ +struct page *kvm_dirty_ring_get_page(struct kvm_dirty_ring *ring, u32 offset); + +void kvm_dirty_ring_free(struct kvm_dirty_ring *ring); +bool kvm_dirty_ring_soft_full(struct kvm_dirty_ring *ring); + +#endif diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index cbd633ece959..c96161c6a0c9 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -34,6 +34,7 @@ #include #include +#include #ifndef KVM_MAX_VCPU_ID #define KVM_MAX_VCPU_ID KVM_MAX_VCPUS @@ -321,6 +322,7 @@ struct kvm_vcpu { bool ready; struct kvm_vcpu_arch arch; struct dentry *debugfs_dentry; + struct kvm_dirty_ring dirty_ring; }; static inline int kvm_vcpu_exiting_guest_mode(struct kvm_vcpu *vcpu) @@ -502,6 +504,7 @@ struct kvm { struct srcu_struct srcu; struct srcu_struct irq_srcu; pid_t userspace_pid; + u32 dirty_ring_size; }; #define kvm_err(fmt, ...) \ @@ -831,6 +834,8 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm, gfn_t gfn_offset, unsigned long mask); +void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask); + int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log); int kvm_vm_ioctl_clear_dirty_log(struct kvm *kvm, @@ -1409,4 +1414,25 @@ int kvm_vm_create_worker_thread(struct kvm *kvm, kvm_vm_thread_fn_t thread_fn, uintptr_t data, const char *name, struct task_struct **thread_ptr); +/* + * This defines how many reserved entries we want to keep before we + * kick the vcpu to the userspace to avoid dirty ring full. This + * value can be tuned to higher if e.g. PML is enabled on the host. + */ +#define KVM_DIRTY_RING_RSVD_ENTRIES 64 + +/* Max number of entries allowed for each kvm dirty ring */ +#define KVM_DIRTY_RING_MAX_ENTRIES 65536 + +/* + * Arch needs to define these macro after implementing the dirty ring + * feature. KVM_DIRTY_LOG_PAGE_OFFSET should be defined as the + * starting page offset of the dirty ring structures, while + * KVM_DIRTY_RING_VERSION should be defined as >=1. By default, this + * feature is off on all archs. + */ +#ifndef KVM_DIRTY_LOG_PAGE_OFFSET +#define KVM_DIRTY_LOG_PAGE_OFFSET 0 +#endif + #endif diff --git a/include/trace/events/kvm.h b/include/trace/events/kvm.h index 2c735a3e6613..3d850997940c 100644 --- a/include/trace/events/kvm.h +++ b/include/trace/events/kvm.h @@ -399,6 +399,84 @@ TRACE_EVENT(kvm_halt_poll_ns, #define trace_kvm_halt_poll_ns_shrink(vcpu_id, new, old) \ trace_kvm_halt_poll_ns(false, vcpu_id, new, old) +TRACE_EVENT(kvm_dirty_ring_push, + TP_PROTO(struct kvm_dirty_ring *ring, u32 slot, u64 offset), + TP_ARGS(ring, slot, offset), + + TP_STRUCT__entry( + __field(int, index) + __field(u32, dirty_index) + __field(u32, reset_index) + __field(u32, slot) + __field(u64, offset) + ), + + TP_fast_assign( + __entry->index = ring->index; + __entry->dirty_index = ring->dirty_index; + __entry->reset_index = ring->reset_index; + __entry->slot = slot; + __entry->offset = offset; + ), + + TP_printk("ring %d: dirty 0x%x reset 0x%x " + "slot %u offset 0x%llx (used %u)", + __entry->index, __entry->dirty_index, + __entry->reset_index, __entry->slot, __entry->offset, + __entry->dirty_index - __entry->reset_index) +); + +TRACE_EVENT(kvm_dirty_ring_reset, + TP_PROTO(struct kvm_dirty_ring *ring), + TP_ARGS(ring), + + TP_STRUCT__entry( + __field(int, index) + __field(u32, dirty_index) + __field(u32, reset_index) + ), + + TP_fast_assign( + __entry->index = ring->index; + __entry->dirty_index = ring->dirty_index; + __entry->reset_index = ring->reset_index; + ), + + TP_printk("ring %d: dirty 0x%x reset 0x%x (used %u)", + __entry->index, __entry->dirty_index, __entry->reset_index, + __entry->dirty_index - __entry->reset_index) +); + +TRACE_EVENT(kvm_dirty_ring_waitqueue, + TP_PROTO(bool enter), + TP_ARGS(enter), + + TP_STRUCT__entry( + __field(bool, enter) + ), + + TP_fast_assign( + __entry->enter = enter; + ), + + TP_printk("%s", __entry->enter ? "wait" : "awake") +); + +TRACE_EVENT(kvm_dirty_ring_exit, + TP_PROTO(struct kvm_vcpu *vcpu), + TP_ARGS(vcpu), + + TP_STRUCT__entry( + __field(int, vcpu_id) + ), + + TP_fast_assign( + __entry->vcpu_id = vcpu->vcpu_id; + ), + + TP_printk("vcpu %d", __entry->vcpu_id) +); + #endif /* _TRACE_KVM_MAIN_H */ /* This part must be outside protection */ diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index f0a16b4adbbd..df4a1700ff1e 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -236,6 +236,7 @@ struct kvm_hyperv_exit { #define KVM_EXIT_IOAPIC_EOI 26 #define KVM_EXIT_HYPERV 27 #define KVM_EXIT_ARM_NISV 28 +#define KVM_EXIT_DIRTY_RING_FULL 29 /* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */ @@ -247,6 +248,13 @@ struct kvm_hyperv_exit { /* Encounter unexpected vm-exit reason */ #define KVM_INTERNAL_ERROR_UNEXPECTED_EXIT_REASON 4 +struct kvm_dirty_ring_indices { + __u32 avail_index; /* set by kernel */ + __u32 padding1; + __u32 fetch_index; /* set by userspace */ + __u32 padding2; +}; + /* for KVM_RUN, returned by mmap(vcpu_fd, offset=0) */ struct kvm_run { /* in */ @@ -421,6 +429,8 @@ struct kvm_run { struct kvm_sync_regs regs; char padding[SYNC_REGS_SIZE_BYTES]; } s; + + struct kvm_dirty_ring_indices vcpu_ring_indices; }; /* for KVM_REGISTER_COALESCED_MMIO / KVM_UNREGISTER_COALESCED_MMIO */ @@ -1009,6 +1019,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_PPC_GUEST_DEBUG_SSTEP 176 #define KVM_CAP_ARM_NISV_TO_USER 177 #define KVM_CAP_ARM_INJECT_EXT_DABT 178 +#define KVM_CAP_DIRTY_LOG_RING 179 #ifdef KVM_CAP_IRQ_ROUTING @@ -1473,6 +1484,9 @@ struct kvm_enc_region { /* Available with KVM_CAP_ARM_SVE */ #define KVM_ARM_VCPU_FINALIZE _IOW(KVMIO, 0xc2, int) +/* Available with KVM_CAP_DIRTY_LOG_RING */ +#define KVM_RESET_DIRTY_RINGS _IO(KVMIO, 0xc3) + /* Secure Encrypted Virtualization command */ enum sev_cmd_id { /* Guest initialization commands */ @@ -1623,4 +1637,23 @@ struct kvm_hyperv_eventfd { #define KVM_HYPERV_CONN_ID_MASK 0x00ffffff #define KVM_HYPERV_EVENTFD_DEASSIGN (1 << 0) +/* + * The following are the requirements for supporting dirty log ring + * (by enabling KVM_DIRTY_LOG_PAGE_OFFSET). + * + * 1. Memory accesses by KVM should call kvm_vcpu_write_* instead + * of kvm_write_* so that the global dirty ring is not filled up + * too quickly. + * 2. kvm_arch_mmu_enable_log_dirty_pt_masked should be defined for + * enabling dirty logging. + * 3. There should not be a separate step to synchronize hardware + * dirty bitmap with KVM's. + */ + +struct kvm_dirty_gfn { + __u32 pad; + __u32 slot; + __u64 offset; +}; + #endif /* __LINUX_KVM_H */ diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c new file mode 100644 index 000000000000..67ec5bbc21c0 --- /dev/null +++ b/virt/kvm/dirty_ring.c @@ -0,0 +1,162 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * KVM dirty ring implementation + * + * Copyright 2019 Red Hat, Inc. + */ +#include +#include +#include +#include +#include + +int __weak kvm_cpu_dirty_log_size(void) +{ + return 0; +} + +u32 kvm_dirty_ring_get_rsvd_entries(void) +{ + return KVM_DIRTY_RING_RSVD_ENTRIES + kvm_cpu_dirty_log_size(); +} + +static u32 kvm_dirty_ring_used(struct kvm_dirty_ring *ring) +{ + return READ_ONCE(ring->dirty_index) - READ_ONCE(ring->reset_index); +} + +bool kvm_dirty_ring_soft_full(struct kvm_dirty_ring *ring) +{ + return kvm_dirty_ring_used(ring) >= ring->soft_limit; +} + +bool kvm_dirty_ring_full(struct kvm_dirty_ring *ring) +{ + return kvm_dirty_ring_used(ring) >= ring->size; +} + +struct kvm_dirty_ring *kvm_dirty_ring_get(struct kvm *kvm) +{ + struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); + + WARN_ON_ONCE(vcpu->kvm != kvm); + + return &vcpu->dirty_ring; +} + +int kvm_dirty_ring_alloc(struct kvm_dirty_ring *ring, + struct kvm_dirty_ring_indices *indices, + int index, u32 size) +{ + ring->dirty_gfns = vmalloc(size); + if (!ring->dirty_gfns) + return -ENOMEM; + memset(ring->dirty_gfns, 0, size); + + ring->size = size / sizeof(struct kvm_dirty_gfn); + ring->soft_limit = ring->size - kvm_dirty_ring_get_rsvd_entries(); + ring->dirty_index = 0; + ring->reset_index = 0; + ring->index = index; + ring->indices = indices; + + return 0; +} + +int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring) +{ + u32 cur_slot, next_slot; + u64 cur_offset, next_offset; + unsigned long mask; + u32 fetch; + int count = 0; + struct kvm_dirty_gfn *entry; + struct kvm_dirty_ring_indices *indices = ring->indices; + bool first_round = true; + + fetch = READ_ONCE(indices->fetch_index); + + /* + * Note that fetch_index is written by the userspace, which + * should not be trusted. If this happens, then it's probably + * that the userspace has written a wrong fetch_index. + */ + if (fetch - ring->reset_index > ring->size) + return -EINVAL; + + if (fetch == ring->reset_index) + return 0; + + /* This is only needed to make compilers happy */ + cur_slot = cur_offset = mask = 0; + while (ring->reset_index != fetch) { + entry = &ring->dirty_gfns[ring->reset_index & (ring->size - 1)]; + next_slot = READ_ONCE(entry->slot); + next_offset = READ_ONCE(entry->offset); + ring->reset_index++; + count++; + /* + * Try to coalesce the reset operations when the guest is + * scanning pages in the same slot. + */ + if (!first_round && next_slot == cur_slot) { + s64 delta = next_offset - cur_offset; + + if (delta >= 0 && delta < BITS_PER_LONG) { + mask |= 1ull << delta; + continue; + } + + /* Backwards visit, careful about overflows! */ + if (delta > -BITS_PER_LONG && delta < 0 && + (mask << -delta >> -delta) == mask) { + cur_offset = next_offset; + mask = (mask << -delta) | 1; + continue; + } + } + kvm_reset_dirty_gfn(kvm, cur_slot, cur_offset, mask); + cur_slot = next_slot; + cur_offset = next_offset; + mask = 1; + first_round = false; + } + kvm_reset_dirty_gfn(kvm, cur_slot, cur_offset, mask); + + trace_kvm_dirty_ring_reset(ring); + + return count; +} + +void kvm_dirty_ring_push(struct kvm_dirty_ring *ring, u32 slot, u64 offset) +{ + struct kvm_dirty_gfn *entry; + struct kvm_dirty_ring_indices *indices = ring->indices; + + /* It should never get full */ + WARN_ON_ONCE(kvm_dirty_ring_full(ring)); + + entry = &ring->dirty_gfns[ring->dirty_index & (ring->size - 1)]; + entry->slot = slot; + entry->offset = offset; + /* + * Make sure the data is filled in before we publish this to + * the userspace program. There's no paired kernel-side reader. + */ + smp_wmb(); + ring->dirty_index++; + WRITE_ONCE(indices->avail_index, ring->dirty_index); + + trace_kvm_dirty_ring_push(ring, slot, offset); +} + +struct page *kvm_dirty_ring_get_page(struct kvm_dirty_ring *ring, u32 offset) +{ + return vmalloc_to_page((void *)ring->dirty_gfns + offset * PAGE_SIZE); +} + +void kvm_dirty_ring_free(struct kvm_dirty_ring *ring) +{ + vfree(ring->dirty_gfns); + ring->dirty_gfns = NULL; +} diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 5bbd8b8730fa..5e36792e15ae 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -64,6 +64,8 @@ #define CREATE_TRACE_POINTS #include +#include + /* Worst case buffer size needed for holding an integer. */ #define ITOA_MAX_LEN 12 @@ -357,11 +359,22 @@ int kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id) vcpu->preempted = false; vcpu->ready = false; + if (kvm->dirty_ring_size) { + r = kvm_dirty_ring_alloc(&vcpu->dirty_ring, + &vcpu->run->vcpu_ring_indices, + id, kvm->dirty_ring_size); + if (r) + goto fail_free_run; + } + r = kvm_arch_vcpu_init(vcpu); if (r < 0) - goto fail_free_run; + goto fail_free_ring; return 0; +fail_free_ring: + if (kvm->dirty_ring_size) + kvm_dirty_ring_free(&vcpu->dirty_ring); fail_free_run: free_page((unsigned long)vcpu->run); fail: @@ -379,6 +392,8 @@ void kvm_vcpu_uninit(struct kvm_vcpu *vcpu) put_pid(rcu_dereference_protected(vcpu->pid, 1)); kvm_arch_vcpu_uninit(vcpu); free_page((unsigned long)vcpu->run); + if (vcpu->kvm->dirty_ring_size) + kvm_dirty_ring_free(&vcpu->dirty_ring); } EXPORT_SYMBOL_GPL(kvm_vcpu_uninit); @@ -2284,8 +2299,13 @@ static void mark_page_dirty_in_slot(struct kvm *kvm, { if (memslot && memslot->dirty_bitmap) { unsigned long rel_gfn = gfn - memslot->base_gfn; + u32 slot = (memslot->as_id << 16) | memslot->id; - set_bit_le(rel_gfn, memslot->dirty_bitmap); + if (kvm->dirty_ring_size) + kvm_dirty_ring_push(kvm_dirty_ring_get(kvm), + slot, rel_gfn); + else + set_bit_le(rel_gfn, memslot->dirty_bitmap); } } @@ -2632,6 +2652,16 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me, bool yield_to_kernel_mode) } EXPORT_SYMBOL_GPL(kvm_vcpu_on_spin); +static bool kvm_page_in_dirty_ring(struct kvm *kvm, unsigned long pgoff) +{ + if (!KVM_DIRTY_LOG_PAGE_OFFSET) + return false; + + return (pgoff >= KVM_DIRTY_LOG_PAGE_OFFSET) && + (pgoff < KVM_DIRTY_LOG_PAGE_OFFSET + + kvm->dirty_ring_size / PAGE_SIZE); +} + static vm_fault_t kvm_vcpu_fault(struct vm_fault *vmf) { struct kvm_vcpu *vcpu = vmf->vma->vm_file->private_data; @@ -2647,6 +2677,10 @@ static vm_fault_t kvm_vcpu_fault(struct vm_fault *vmf) else if (vmf->pgoff == KVM_COALESCED_MMIO_PAGE_OFFSET) page = virt_to_page(vcpu->kvm->coalesced_mmio_ring); #endif + else if (kvm_page_in_dirty_ring(vcpu->kvm, vmf->pgoff)) + page = kvm_dirty_ring_get_page( + &vcpu->dirty_ring, + vmf->pgoff - KVM_DIRTY_LOG_PAGE_OFFSET); else return kvm_arch_vcpu_fault(vcpu, vmf); get_page(page); @@ -2660,6 +2694,15 @@ static const struct vm_operations_struct kvm_vcpu_vm_ops = { static int kvm_vcpu_mmap(struct file *file, struct vm_area_struct *vma) { + struct kvm_vcpu *vcpu = file->private_data; + unsigned long pages = (vma->vm_end - vma->vm_start) >> PAGE_SHIFT; + + /* If to map any writable page within dirty ring, fail it */ + if ((kvm_page_in_dirty_ring(vcpu->kvm, vma->vm_pgoff) || + kvm_page_in_dirty_ring(vcpu->kvm, vma->vm_pgoff + pages - 1)) && + vma->vm_flags & VM_WRITE) + return -EINVAL; + vma->vm_ops = &kvm_vcpu_vm_ops; return 0; } @@ -3242,12 +3285,97 @@ static long kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg) #endif case KVM_CAP_NR_MEMSLOTS: return KVM_USER_MEM_SLOTS; + case KVM_CAP_DIRTY_LOG_RING: +#ifdef CONFIG_X86 + return KVM_DIRTY_RING_MAX_ENTRIES; +#else + return 0; +#endif default: break; } return kvm_vm_ioctl_check_extension(kvm, arg); } +void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask) +{ + struct kvm_memory_slot *memslot; + int as_id, id; + + as_id = slot >> 16; + id = (u16)slot; + if (as_id >= KVM_ADDRESS_SPACE_NUM || id >= KVM_USER_MEM_SLOTS) + return; + + memslot = id_to_memslot(__kvm_memslots(kvm, as_id), id); + if (offset >= memslot->npages) + return; + + spin_lock(&kvm->mmu_lock); + kvm_arch_mmu_enable_log_dirty_pt_masked(kvm, memslot, offset, mask); + spin_unlock(&kvm->mmu_lock); +} + +static int kvm_vm_ioctl_enable_dirty_log_ring(struct kvm *kvm, u32 size) +{ + int r; + + if (!KVM_DIRTY_LOG_PAGE_OFFSET) + return -EINVAL; + + /* the size should be power of 2 */ + if (!size || (size & (size - 1))) + return -EINVAL; + + /* Should be bigger to keep the reserved entries, or a page */ + if (size < kvm_dirty_ring_get_rsvd_entries() * + sizeof(struct kvm_dirty_gfn) || size < PAGE_SIZE) + return -EINVAL; + + if (size > KVM_DIRTY_RING_MAX_ENTRIES * + sizeof(struct kvm_dirty_gfn)) + return -E2BIG; + + /* We only allow it to set once */ + if (kvm->dirty_ring_size) + return -EINVAL; + + mutex_lock(&kvm->lock); + + if (kvm->created_vcpus) { + /* We don't allow to change this value after vcpu created */ + r = -EINVAL; + } else { + kvm->dirty_ring_size = size; + r = 0; + } + + mutex_unlock(&kvm->lock); + return r; +} + +static int kvm_vm_ioctl_reset_dirty_pages(struct kvm *kvm) +{ + int i; + struct kvm_vcpu *vcpu; + int cleared = 0; + + if (!kvm->dirty_ring_size) + return -EINVAL; + + mutex_lock(&kvm->slots_lock); + + kvm_for_each_vcpu(i, vcpu, kvm) + cleared += kvm_dirty_ring_reset(vcpu->kvm, &vcpu->dirty_ring); + + mutex_unlock(&kvm->slots_lock); + + if (cleared) + kvm_flush_remote_tlbs(kvm); + + return cleared; +} + int __attribute__((weak)) kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { @@ -3265,6 +3393,8 @@ static int kvm_vm_ioctl_enable_cap_generic(struct kvm *kvm, kvm->manual_dirty_log_protect = cap->args[0]; return 0; #endif + case KVM_CAP_DIRTY_LOG_RING: + return kvm_vm_ioctl_enable_dirty_log_ring(kvm, cap->args[0]); default: return kvm_vm_ioctl_enable_cap(kvm, cap); } @@ -3452,6 +3582,9 @@ static long kvm_vm_ioctl(struct file *filp, case KVM_CHECK_EXTENSION: r = kvm_vm_ioctl_check_extension_generic(kvm, arg); break; + case KVM_RESET_DIRTY_RINGS: + r = kvm_vm_ioctl_reset_dirty_pages(kvm); + break; default: r = kvm_arch_vm_ioctl(filp, ioctl, arg); } From patchwork Thu Jan 9 14:57:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325915 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0BEE92A for ; Thu, 9 Jan 2020 14:59:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9684B2077C for ; Thu, 9 Jan 2020 14:59:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="J/xUej8s" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732035AbgAIO7P (ORCPT ); Thu, 9 Jan 2020 09:59:15 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:25537 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732045AbgAIO6O (ORCPT ); Thu, 9 Jan 2020 09:58:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581894; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2hsGT2puS2M7nFUR+/hmDKhyO+EnbA4DSJk9zAXszck=; b=J/xUej8sjJmpErfsXQ/qsgmnYYlaSwX9l5rnqPIxJMeHJdZuCXbqPRt5Af11LRV4CWuiSk 7X6J1JKgX4WNjxVml/IJoEdj7HDEcjAbHj/XLhuMlZo0t43oxZoQbDZKrswhq6yF0Ty6WO xljRHlOykQ1bZQTG7LLl5UbCZRu7Iw4= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-168-w3L6pAt1N7GQYoAnEMsNaQ-1; Thu, 09 Jan 2020 09:58:13 -0500 X-MC-Unique: w3L6pAt1N7GQYoAnEMsNaQ-1 Received: by mail-qt1-f200.google.com with SMTP id l56so4338694qtk.11 for ; Thu, 09 Jan 2020 06:58:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2hsGT2puS2M7nFUR+/hmDKhyO+EnbA4DSJk9zAXszck=; b=JnMFVnHm3w3uojNizCIE9KYyYxy3WKHKCN1f2LI7Rex/Qb6xyyyVZ3bPg04csxsijI X3Q/QD3EbYs5p1OMyTNpOCOtT3edhICXsK2Yvyk+Ile7jrkn+ixHX896v2vVTl+BQscP SbvhAA5ymmqy/UJI1z2H1YEDLFTB8rBXeD+BYjEoFZs/DuC6M6PK0kpxaKmn6RofbxLD X/J/vwDEu5afL5yXRkhTrQEM9TI6mi6f3MAF9iLlAS65kWRP4B3vdbAD2Ue0/95E1sDS eFbFOXjVzNXkVTVCJTiJxqcWz6OYNdwwayuj32qbaalhh/KSkctmsT65wLO6Rzru65qd IeHQ== X-Gm-Message-State: APjAAAWX3SUHl0S1/fGVZafvglKD6QRMLd0hHNZl98we1x0H8gE8LspP XscvmgTezFXP2kH4OoKchFO3t6bPvxLaZXgA6wKlKahT15JBvQr76grEp+cN20vvWdzeo2PoN/5 5VrBa1fS19SEZ X-Received: by 2002:a37:e507:: with SMTP id e7mr10127467qkg.358.1578581888963; Thu, 09 Jan 2020 06:58:08 -0800 (PST) X-Google-Smtp-Source: APXvYqz0H7o6nlbN/ohUkbX/oqc+8r0t97p+EDwDPG2HIfZBZWc9Yu+XnrfPuT1t0LiHOiml2tX/xQ== X-Received: by 2002:a37:e507:: with SMTP id e7mr10127431qkg.358.1578581888713; Thu, 09 Jan 2020 06:58:08 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.58.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:58:07 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 13/21] KVM: Make dirty ring exclusive to dirty bitmap log Date: Thu, 9 Jan 2020 09:57:21 -0500 Message-Id: <20200109145729.32898-14-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org There's no good reason to use both the dirty bitmap logging and the new dirty ring buffer to track dirty bits. We should be able to even support both of them at the same time, but it could complicate things which could actually help little. Let's simply make it the rule before we enable dirty ring on any arch, that we don't allow these two interfaces to be used together. The big world switch would be KVM_CAP_DIRTY_LOG_RING capability enablement. That's where we'll switch from the default dirty logging way to the dirty ring way. As long as kvm->dirty_ring_size is setup correctly, we'll once and for all switch to the dirty ring buffer mode for the current virtual machine. Signed-off-by: Peter Xu --- Documentation/virt/kvm/api.txt | 7 +++++++ virt/kvm/kvm_main.c | 12 ++++++++++++ 2 files changed, 19 insertions(+) diff --git a/Documentation/virt/kvm/api.txt b/Documentation/virt/kvm/api.txt index 708c3e0f7eae..be176d1dd91f 100644 --- a/Documentation/virt/kvm/api.txt +++ b/Documentation/virt/kvm/api.txt @@ -5485,3 +5485,10 @@ all the existing dirty gfns are flushed to the dirty rings. If one of the ring buffers is full, the guest will exit to userspace with the exit reason set to KVM_EXIT_DIRTY_LOG_FULL, and the KVM_RUN ioctl will return to userspace with zero. + +NOTE: the KVM_CAP_DIRTY_LOG_RING capability and the new ioctl +KVM_RESET_DIRTY_RINGS are exclusive to the existing KVM_GET_DIRTY_LOG +interface. After enabling KVM_CAP_DIRTY_LOG_RING with an acceptable +dirty ring size, the virtual machine will switch to the dirty ring +tracking mode, and KVM_GET_DIRTY_LOG, KVM_CLEAR_DIRTY_LOG ioctls will +stop working. diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 5e36792e15ae..f0f766183cb2 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1211,6 +1211,10 @@ int kvm_get_dirty_log(struct kvm *kvm, unsigned long n; unsigned long any = 0; + /* Dirty ring tracking is exclusive to dirty log tracking */ + if (kvm->dirty_ring_size) + return -EINVAL; + as_id = log->slot >> 16; id = (u16)log->slot; if (as_id >= KVM_ADDRESS_SPACE_NUM || id >= KVM_USER_MEM_SLOTS) @@ -1268,6 +1272,10 @@ int kvm_get_dirty_log_protect(struct kvm *kvm, unsigned long *dirty_bitmap; unsigned long *dirty_bitmap_buffer; + /* Dirty ring tracking is exclusive to dirty log tracking */ + if (kvm->dirty_ring_size) + return -EINVAL; + as_id = log->slot >> 16; id = (u16)log->slot; if (as_id >= KVM_ADDRESS_SPACE_NUM || id >= KVM_USER_MEM_SLOTS) @@ -1339,6 +1347,10 @@ int kvm_clear_dirty_log_protect(struct kvm *kvm, unsigned long *dirty_bitmap; unsigned long *dirty_bitmap_buffer; + /* Dirty ring tracking is exclusive to dirty log tracking */ + if (kvm->dirty_ring_size) + return -EINVAL; + as_id = log->slot >> 16; id = (u16)log->slot; if (as_id >= KVM_ADDRESS_SPACE_NUM || id >= KVM_USER_MEM_SLOTS) From patchwork Thu Jan 9 14:57:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325913 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EE0B292A for ; Thu, 9 Jan 2020 14:59:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CC79420721 for ; Thu, 9 Jan 2020 14:59:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="dZ4PuZ24" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732071AbgAIO7J (ORCPT ); Thu, 9 Jan 2020 09:59:09 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:50348 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732058AbgAIO6R (ORCPT ); Thu, 9 Jan 2020 09:58:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581896; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qo8B0RRH3HwVIIYkQ204ghyrR8Ipv7FatZgkB5Hr2Ro=; b=dZ4PuZ24tAZuRto9B9cOLtNO1lYtFuFB8OuBfaQSE6Urc4/1fJufPEIe+FMt6njKU/fV8A zpXw36BLuzq3DFy8u1wpisehjKohFCII/7wpLjcnsniEeo+XSsaBCSiKXC+Jc8DPQANzbr XZrIU4/yt0v+FNWgbCzrP9Fu1PyfZII= Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-296-ZYyQCIdFNHa9W7wnw5IaMw-1; Thu, 09 Jan 2020 09:58:14 -0500 X-MC-Unique: ZYyQCIdFNHa9W7wnw5IaMw-1 Received: by mail-qk1-f198.google.com with SMTP id 24so4262339qka.16 for ; Thu, 09 Jan 2020 06:58:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qo8B0RRH3HwVIIYkQ204ghyrR8Ipv7FatZgkB5Hr2Ro=; b=c+BIZXbMCUjWmVY1hzNrp2BfpEXJGTUmuLnVGB5hd0RgGlc5WS/u10gEX/E4+7QMpS qWBW/MwJGwFGk15tiq0wPEkSDzCxCBKvXS2URHPMJ3Ojwm6Ae3Sm95aXH8OsiYOqsvSU gtZukGSHxrs+WwDqpgOv/F8MgrV/oFvRhVqX4p5rKKSZ+UiulWUHnRpOXL/0lt3FvfPk 4pTX9cuxsb+qpAVuIiotJlBc0nc6IjRFVgFAoklzggjAmxSgBHdn2U1IAVd9uWfdHw8F NGDUPNO4bnr1u3ovaaU0AwNjvqzoZ7BBW9Busw5UPORloSZc9JN0qR9chAPrYeN+G4H7 AuNA== X-Gm-Message-State: APjAAAXA72qGnt3LzkMPl6st9cr3xyrVR5EHsacg7vVYrG1ruV9aZwX8 sT/xXoxD79Aom0H2bHRaqWUZKaB3tVkQ+iexUfljXeJxFXHjZf3WLCJQOblUaXQTHs3Lg5VFGCz ohuT40D/uJvo4 X-Received: by 2002:ad4:55ec:: with SMTP id bu12mr9341183qvb.107.1578581894160; Thu, 09 Jan 2020 06:58:14 -0800 (PST) X-Google-Smtp-Source: APXvYqxap7TJb0aJMeIGMtE0Ap5zzFfcOhoJTtDvs6UIukIGtqjETcSsqkFLZb0v4jXm6uqUuxkLFg== X-Received: by 2002:ad4:55ec:: with SMTP id bu12mr9341162qvb.107.1578581893880; Thu, 09 Jan 2020 06:58:13 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.58.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:58:12 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 14/21] KVM: Don't allocate dirty bitmap if dirty ring is enabled Date: Thu, 9 Jan 2020 09:57:22 -0500 Message-Id: <20200109145729.32898-15-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Because kvm dirty rings and kvm dirty log is used in an exclusive way, Let's avoid creating the dirty_bitmap when kvm dirty ring is enabled. At the meantime, since the dirty_bitmap will be conditionally created now, we can't use it as a sign of "whether this memory slot enabled dirty tracking". Change users like that to check against the kvm memory slot flags. Note that there still can be chances where the kvm memory slot got its dirty_bitmap allocated, _if_ the memory slots are created before enabling of the dirty rings and at the same time with the dirty tracking capability enabled, they'll still with the dirty_bitmap. However it should not hurt much (e.g., the bitmaps will always be freed if they are there), and the real users normally won't trigger this because dirty bit tracking flag should in most cases only be applied to kvm slots only before migration starts, that should be far latter than kvm initializes (VM starts). Signed-off-by: Peter Xu --- include/linux/kvm_host.h | 5 +++++ virt/kvm/kvm_main.c | 5 +++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index c96161c6a0c9..ab2a169b1264 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -353,6 +353,11 @@ struct kvm_memory_slot { u8 as_id; }; +static inline bool kvm_slot_dirty_track_enabled(struct kvm_memory_slot *slot) +{ + return slot->flags & KVM_MEM_LOG_DIRTY_PAGES; +} + static inline unsigned long kvm_dirty_bitmap_bytes(struct kvm_memory_slot *memslot) { return ALIGN(memslot->npages, BITS_PER_LONG) / 8; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index f0f766183cb2..46da3169944f 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1120,7 +1120,8 @@ int __kvm_set_memory_region(struct kvm *kvm, } /* Allocate page dirty bitmap if needed */ - if ((new.flags & KVM_MEM_LOG_DIRTY_PAGES) && !new.dirty_bitmap) { + if ((new.flags & KVM_MEM_LOG_DIRTY_PAGES) && !new.dirty_bitmap && + !kvm->dirty_ring_size) { if (kvm_create_dirty_bitmap(&new) < 0) goto out_free; } @@ -2309,7 +2310,7 @@ static void mark_page_dirty_in_slot(struct kvm *kvm, struct kvm_memory_slot *memslot, gfn_t gfn) { - if (memslot && memslot->dirty_bitmap) { + if (memslot && kvm_slot_dirty_track_enabled(memslot)) { unsigned long rel_gfn = gfn - memslot->base_gfn; u32 slot = (memslot->as_id << 16) | memslot->id; From patchwork Thu Jan 9 14:57:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325911 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A7F3492A for ; Thu, 9 Jan 2020 14:59:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 86ABC20721 for ; Thu, 9 Jan 2020 14:59:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="X/iJddj+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732032AbgAIO7A (ORCPT ); Thu, 9 Jan 2020 09:59:00 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:44593 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732068AbgAIO6T (ORCPT ); Thu, 9 Jan 2020 09:58:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581898; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v+zy2FHQw9KFVxtHaLqa0r2t8FDa4q1KvvQqvnAapvQ=; b=X/iJddj+B4osbqkETOxgxRJIE9I7GFU6gHQaeDAF9nPY5MflJiyxWukqnbjEpTR2XEyVho q1rfUyWN5iTrEM9fwazu9l1EV4uLGd8tP/b0agECsvvk2sWFRYRH5INfqaTN975R0hlShZ 08t3oiIxJPZeWEjZUQSNleg+KBfZE7k= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-423-4v0d8TVzOGubRUApxHnTZQ-1; Thu, 09 Jan 2020 09:58:17 -0500 X-MC-Unique: 4v0d8TVzOGubRUApxHnTZQ-1 Received: by mail-qk1-f200.google.com with SMTP id 143so4273160qkg.12 for ; Thu, 09 Jan 2020 06:58:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=v+zy2FHQw9KFVxtHaLqa0r2t8FDa4q1KvvQqvnAapvQ=; b=aj6t3mF3PbmPeku6WIiV3bFRpydBLY5HSHTUwABmT8pYwfUyx62vSETYIzA9cTLzom Ej8NNL/sEsuWFN5XKgVvTDvmbxxgE5COCJGBu831m839mAxuuZms9/FC85ddZnNfmSR+ FHNi3DOYcnuoCElugGl6jJHstHD956DYl7e/IK2AnPsd8akjpVH/J53Jn+X4GkPXq11K VDTYNJC9Q/yBPnmFixhlkrGjFqbboufYbVUxcAsSOQ6NrSclZzb5Hq7qdhfnlC0py0CX QfbYlElqk4D30HoWPvL3Uf0EwE0GPVB/KwAty1OOkFSDY136WQQww47KhCqwoeboaxM1 j+6Q== X-Gm-Message-State: APjAAAWjr/7nhCbliHmMCGcUJLfga0/U/6RGraWBB91EbFQ+l/b079tW hunLMEwv4nxHYjk6BHzqJ978RRO9wUFlAzjUIC4q4WM5Po7AxCTIwGcCuLUfcujquthDLj+meUh FJ4cGmvZIppBt X-Received: by 2002:a37:9ace:: with SMTP id c197mr10113190qke.482.1578581895510; Thu, 09 Jan 2020 06:58:15 -0800 (PST) X-Google-Smtp-Source: APXvYqztXceCqR3GxYbBVVwKVyCXfRviIN28zAumlgJGOwqri1GbhE+3hiqmzypgekRZtjwYhslICg== X-Received: by 2002:a37:9ace:: with SMTP id c197mr10113165qke.482.1578581895289; Thu, 09 Jan 2020 06:58:15 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.58.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:58:14 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 15/21] KVM: selftests: Always clear dirty bitmap after iteration Date: Thu, 9 Jan 2020 09:57:23 -0500 Message-Id: <20200109145729.32898-16-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org We don't clear the dirty bitmap before because KVM_GET_DIRTY_LOG will clear it for us before copying the dirty log onto it. However we'd still better to clear it explicitly instead of assuming the kernel will always do it for us. More importantly, in the upcoming dirty ring tests we'll start to fetch dirty pages from a ring buffer, so no one is going to clear the dirty bitmap for us. Signed-off-by: Peter Xu --- tools/testing/selftests/kvm/dirty_log_test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/kvm/dirty_log_test.c b/tools/testing/selftests/kvm/dirty_log_test.c index 5614222a6628..3c0ffd34b3b0 100644 --- a/tools/testing/selftests/kvm/dirty_log_test.c +++ b/tools/testing/selftests/kvm/dirty_log_test.c @@ -197,7 +197,7 @@ static void vm_dirty_log_verify(unsigned long *bmap) page); } - if (test_bit_le(page, bmap)) { + if (test_and_clear_bit_le(page, bmap)) { host_dirty_count++; /* * If the bit is set, the value written onto From patchwork Thu Jan 9 14:57:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325909 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C45BA109A for ; Thu, 9 Jan 2020 14:58:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 985202067D for ; Thu, 9 Jan 2020 14:58:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QApgU3+B" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732147AbgAIO6t (ORCPT ); Thu, 9 Jan 2020 09:58:49 -0500 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:23476 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732069AbgAIO6T (ORCPT ); Thu, 9 Jan 2020 09:58:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581898; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EkI6h8xSN7v+a4eMQahKlRzNqUClc6kri5FkEiWNSSA=; b=QApgU3+BKXSIIzpsfnHfNqtIW0nY2BldtgSGuXmyIbFy0wvl0XE/l9E+CLUrgHVDhjs6D4 HzSdQ49TxB7u8PhJdsjyfNmY55p4092/GsjAUzvHIqgHObdipcwsEMAQV6py/ZL28GO+na SnDmrx7TXzYGnA4fU9D/BNVzrmSOMPM= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-232-47R9hmKfN4mTfDuN8BY3JA-1; Thu, 09 Jan 2020 09:58:17 -0500 X-MC-Unique: 47R9hmKfN4mTfDuN8BY3JA-1 Received: by mail-qt1-f198.google.com with SMTP id b14so4394110qtt.1 for ; Thu, 09 Jan 2020 06:58:17 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EkI6h8xSN7v+a4eMQahKlRzNqUClc6kri5FkEiWNSSA=; b=G2PGJzZrAR+p7DG87bbb4ytO3VETSn0kX/mFr4rPcSXY8o853c4h+z19pBpxECdy1a TnrSaGtuRYehLh/GbXpz79xhFYk3cS3iKRC6sNhDUu2q29ASpVA86exXT1ZHqLbmru5t pNtlkjQzX4qrh9tx1aTUrAiEgIFkn3KPTpIkM5j0hiew9ojI+a/X27l5k6DHEZnv4LXI EMngUCntFWu4rWGZ6Cy7wB3r+t4xZ2DvJ16DbxQ8BkVbm/QDqs3xeuHZuYEfkO42oMhu RlFRNQjo/PF37PYDX1JI3LR9yVlxk4fYh07+N0yyc/s2kuvXjW52j0KI4ufJB70T0Wqj wURw== X-Gm-Message-State: APjAAAV9zJBMxKXJjNZSO0DqGXA5nIVE0uh2kGRlOUTwrCGXK2qH/9MI /R1INna+rh1p85w9uA7L4jzJFDaqzpw7Rz/wFlj/nWDuVq3IWshh53sArJUwtha/j/OOsouK2y1 d1MmrAWmx3gYP X-Received: by 2002:ae9:f442:: with SMTP id z2mr10150768qkl.130.1578581896686; Thu, 09 Jan 2020 06:58:16 -0800 (PST) X-Google-Smtp-Source: APXvYqzBAarLMa55uVokrirkTNLb6eELDuIHQGq/dU1S1sfC9y2eLW5K4vaMv9gn/0Ij3PCki0kUlw== X-Received: by 2002:ae9:f442:: with SMTP id z2mr10150744qkl.130.1578581896459; Thu, 09 Jan 2020 06:58:16 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.58.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:58:16 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 16/21] KVM: selftests: Sync uapi/linux/kvm.h to tools/ Date: Thu, 9 Jan 2020 09:57:24 -0500 Message-Id: <20200109145729.32898-17-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This will be needed to extend the kvm selftest program. Signed-off-by: Peter Xu --- tools/include/uapi/linux/kvm.h | 38 ++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h index f0a16b4adbbd..d2300a3cfbf0 100644 --- a/tools/include/uapi/linux/kvm.h +++ b/tools/include/uapi/linux/kvm.h @@ -236,6 +236,7 @@ struct kvm_hyperv_exit { #define KVM_EXIT_IOAPIC_EOI 26 #define KVM_EXIT_HYPERV 27 #define KVM_EXIT_ARM_NISV 28 +#define KVM_EXIT_DIRTY_RING_FULL 29 /* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */ @@ -247,6 +248,13 @@ struct kvm_hyperv_exit { /* Encounter unexpected vm-exit reason */ #define KVM_INTERNAL_ERROR_UNEXPECTED_EXIT_REASON 4 +struct kvm_dirty_ring_indices { + __u32 avail_index; /* set by kernel */ + __u32 padding1; + __u32 fetch_index; /* set by userspace */ + __u32 padding2; +}; + /* for KVM_RUN, returned by mmap(vcpu_fd, offset=0) */ struct kvm_run { /* in */ @@ -421,6 +429,13 @@ struct kvm_run { struct kvm_sync_regs regs; char padding[SYNC_REGS_SIZE_BYTES]; } s; + + struct kvm_dirty_ring_indices vcpu_ring_indices; +}; + +/* Returned by mmap(kvm->fd, offset=0) */ +struct kvm_vm_run { + struct kvm_dirty_ring_indices vm_ring_indices; }; /* for KVM_REGISTER_COALESCED_MMIO / KVM_UNREGISTER_COALESCED_MMIO */ @@ -1009,6 +1024,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_PPC_GUEST_DEBUG_SSTEP 176 #define KVM_CAP_ARM_NISV_TO_USER 177 #define KVM_CAP_ARM_INJECT_EXT_DABT 178 +#define KVM_CAP_DIRTY_LOG_RING 179 #ifdef KVM_CAP_IRQ_ROUTING @@ -1473,6 +1489,9 @@ struct kvm_enc_region { /* Available with KVM_CAP_ARM_SVE */ #define KVM_ARM_VCPU_FINALIZE _IOW(KVMIO, 0xc2, int) +/* Available with KVM_CAP_DIRTY_LOG_RING */ +#define KVM_RESET_DIRTY_RINGS _IO(KVMIO, 0xc3) + /* Secure Encrypted Virtualization command */ enum sev_cmd_id { /* Guest initialization commands */ @@ -1623,4 +1642,23 @@ struct kvm_hyperv_eventfd { #define KVM_HYPERV_CONN_ID_MASK 0x00ffffff #define KVM_HYPERV_EVENTFD_DEASSIGN (1 << 0) +/* + * The following are the requirements for supporting dirty log ring + * (by enabling KVM_DIRTY_LOG_PAGE_OFFSET). + * + * 1. Memory accesses by KVM should call kvm_vcpu_write_* instead + * of kvm_write_* so that the global dirty ring is not filled up + * too quickly. + * 2. kvm_arch_mmu_enable_log_dirty_pt_masked should be defined for + * enabling dirty logging. + * 3. There should not be a separate step to synchronize hardware + * dirty bitmap with KVM's. + */ + +struct kvm_dirty_gfn { + __u32 pad; + __u32 slot; + __u64 offset; +}; + #endif /* __LINUX_KVM_H */ From patchwork Thu Jan 9 14:57:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325897 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 69E3592A for ; Thu, 9 Jan 2020 14:58:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 34D832072E for ; Thu, 9 Jan 2020 14:58:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BGQRtJa5" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732088AbgAIO6W (ORCPT ); Thu, 9 Jan 2020 09:58:22 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:57682 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732082AbgAIO6V (ORCPT ); Thu, 9 Jan 2020 09:58:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581900; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WdC8RACEBXWXiNKz4cG4gkf1SXOTtLD6gRrHtU5a1W8=; b=BGQRtJa5/m3ZUYI4ivjP6EH+IslHx3wUD/QIHE7LMsILEsIQ/3TcTE/cj3t/Jiy3y7/eiQ jDnEdFtrkwJMb9z6L9Hnhde4D2MfekgmECQjgjDWWdnpfySoeWFhawl4LTUJs9MlIohFcx +0ZG+ImCXhjNon0lfW5AjjckjsOVZF4= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-432-sY4jGDRDPYCjvcRgzbT2dg-1; Thu, 09 Jan 2020 09:58:19 -0500 X-MC-Unique: sY4jGDRDPYCjvcRgzbT2dg-1 Received: by mail-qv1-f71.google.com with SMTP id di5so4299521qvb.3 for ; Thu, 09 Jan 2020 06:58:19 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WdC8RACEBXWXiNKz4cG4gkf1SXOTtLD6gRrHtU5a1W8=; b=NREtpGbr1nc8qPUJ1ryWK7PiAzxx/6CIqmgybC4skjtfQpVaCMpiN1aGKFQLRtcSTG wXBBiUJKgaCnQj+/6LaSV3fa/zkYLbe//0ESwtY55pG+pBhtZzEOmCIAxgDvhjAB7k7j lkzJIxgKC2/PCg9AmQBUyl1mSrU59eKloGNmUFJz3T/sQfF2jCNr9bc0BiyIV30pBCSH 6pqjiba+rnb0Qsyu0eUTeba04MsXg/dG+0zumI/uhG98tkhaQNnmJbEsRrEi0vDCKiy+ fukHpe8Jtp3be5VHxtinjy5OhxNLKxBOKciEtiR+ydfdPlOPMn8e1WcAYDmUTQvxCap1 S53A== X-Gm-Message-State: APjAAAW54IXwl9cI/Q/oTn5wnzoIeMab5sA/siKX54xwjgH8RNDMu2yb SiJJJ8c391wWkF4EJbtyIPYiodrvOioIi/YrU1FpopCLKc9+0MxniH+d7Qnc021pWdJxBfrummc HnZXxycx9iq3T X-Received: by 2002:ad4:42d1:: with SMTP id f17mr9221061qvr.30.1578581898161; Thu, 09 Jan 2020 06:58:18 -0800 (PST) X-Google-Smtp-Source: APXvYqzhpNurUj8GRzeQ3Gop8kY8cTPv8GUJPgwCzaHpfOsS5/4K7kMszLdKdzN4qHQChEOhvWF+YQ== X-Received: by 2002:ad4:42d1:: with SMTP id f17mr9221042qvr.30.1578581897803; Thu, 09 Jan 2020 06:58:17 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.58.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:58:17 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 17/21] KVM: selftests: Use a single binary for dirty/clear log test Date: Thu, 9 Jan 2020 09:57:25 -0500 Message-Id: <20200109145729.32898-18-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Remove the clear_dirty_log test, instead merge it into the existing dirty_log_test. It should be cleaner to use this single binary to do both tests, also it's a preparation for the upcoming dirty ring test. The default test will still be the dirty_log test. To run the clear dirty log test, we need to specify "-M clear-log". Signed-off-by: Peter Xu --- tools/testing/selftests/kvm/Makefile | 2 - .../selftests/kvm/clear_dirty_log_test.c | 2 - tools/testing/selftests/kvm/dirty_log_test.c | 131 +++++++++++++++--- 3 files changed, 110 insertions(+), 25 deletions(-) delete mode 100644 tools/testing/selftests/kvm/clear_dirty_log_test.c diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index 3138a916574a..130a7b1c7ad6 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -26,11 +26,9 @@ TEST_GEN_PROGS_x86_64 += x86_64/vmx_dirty_log_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_set_nested_state_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test TEST_GEN_PROGS_x86_64 += x86_64/xss_msr_test -TEST_GEN_PROGS_x86_64 += clear_dirty_log_test TEST_GEN_PROGS_x86_64 += dirty_log_test TEST_GEN_PROGS_x86_64 += kvm_create_max_vcpus -TEST_GEN_PROGS_aarch64 += clear_dirty_log_test TEST_GEN_PROGS_aarch64 += dirty_log_test TEST_GEN_PROGS_aarch64 += kvm_create_max_vcpus diff --git a/tools/testing/selftests/kvm/clear_dirty_log_test.c b/tools/testing/selftests/kvm/clear_dirty_log_test.c deleted file mode 100644 index 749336937d37..000000000000 --- a/tools/testing/selftests/kvm/clear_dirty_log_test.c +++ /dev/null @@ -1,2 +0,0 @@ -#define USE_CLEAR_DIRTY_LOG -#include "dirty_log_test.c" diff --git a/tools/testing/selftests/kvm/dirty_log_test.c b/tools/testing/selftests/kvm/dirty_log_test.c index 3c0ffd34b3b0..a8ae8c0042a8 100644 --- a/tools/testing/selftests/kvm/dirty_log_test.c +++ b/tools/testing/selftests/kvm/dirty_log_test.c @@ -128,6 +128,66 @@ static uint64_t host_dirty_count; static uint64_t host_clear_count; static uint64_t host_track_next_count; +enum log_mode_t { + /* Only use KVM_GET_DIRTY_LOG for logging */ + LOG_MODE_DIRTY_LOG = 0, + + /* Use both KVM_[GET|CLEAR]_DIRTY_LOG for logging */ + LOG_MODE_CLERA_LOG = 1, + + LOG_MODE_NUM, +}; + +/* Mode of logging. Default is LOG_MODE_DIRTY_LOG */ +static enum log_mode_t host_log_mode; + +static void clear_log_create_vm_done(struct kvm_vm *vm) +{ + struct kvm_enable_cap cap = {}; + + if (!kvm_check_cap(KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2)) { + fprintf(stderr, "KVM_CLEAR_DIRTY_LOG not available, skipping tests\n"); + exit(KSFT_SKIP); + } + + cap.cap = KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2; + cap.args[0] = 1; + vm_enable_cap(vm, &cap); +} + +static void dirty_log_collect_dirty_pages(struct kvm_vm *vm, int slot, + void *bitmap, uint32_t num_pages) +{ + kvm_vm_get_dirty_log(vm, slot, bitmap); +} + +static void clear_log_collect_dirty_pages(struct kvm_vm *vm, int slot, + void *bitmap, uint32_t num_pages) +{ + kvm_vm_get_dirty_log(vm, slot, bitmap); + kvm_vm_clear_dirty_log(vm, slot, bitmap, 0, num_pages); +} + +struct log_mode { + const char *name; + /* Hook when the vm creation is done (before vcpu creation) */ + void (*create_vm_done)(struct kvm_vm *vm); + /* Hook to collect the dirty pages into the bitmap provided */ + void (*collect_dirty_pages) (struct kvm_vm *vm, int slot, + void *bitmap, uint32_t num_pages); +} log_modes[LOG_MODE_NUM] = { + { + .name = "dirty-log", + .create_vm_done = NULL, + .collect_dirty_pages = dirty_log_collect_dirty_pages, + }, + { + .name = "clear-log", + .create_vm_done = clear_log_create_vm_done, + .collect_dirty_pages = clear_log_collect_dirty_pages, + }, +}; + /* * We use this bitmap to track some pages that should have its dirty * bit set in the _next_ iteration. For example, if we detected the @@ -137,6 +197,33 @@ static uint64_t host_track_next_count; */ static unsigned long *host_bmap_track; +static void log_modes_dump(void) +{ + int i; + + for (i = 0; i < LOG_MODE_NUM; i++) + printf("%s, ", log_modes[i].name); + puts("\b\b \b\b"); +} + +static void log_mode_create_vm_done(struct kvm_vm *vm) +{ + struct log_mode *mode = &log_modes[host_log_mode]; + + if (mode->create_vm_done) + mode->create_vm_done(vm); +} + +static void log_mode_collect_dirty_pages(struct kvm_vm *vm, int slot, + void *bitmap, uint32_t num_pages) +{ + struct log_mode *mode = &log_modes[host_log_mode]; + + TEST_ASSERT(mode->collect_dirty_pages != NULL, + "collect_dirty_pages() is required for any log mode!"); + mode->collect_dirty_pages(vm, slot, bitmap, num_pages); +} + static void generate_random_array(uint64_t *guest_array, uint64_t size) { uint64_t i; @@ -257,6 +344,7 @@ static struct kvm_vm *create_vm(enum vm_guest_mode mode, uint32_t vcpuid, #ifdef __x86_64__ vm_create_irqchip(vm); #endif + log_mode_create_vm_done(vm); vm_vcpu_add_default(vm, vcpuid, guest_code); return vm; } @@ -316,14 +404,6 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations, bmap = bitmap_alloc(host_num_pages); host_bmap_track = bitmap_alloc(host_num_pages); -#ifdef USE_CLEAR_DIRTY_LOG - struct kvm_enable_cap cap = {}; - - cap.cap = KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2; - cap.args[0] = 1; - vm_enable_cap(vm, &cap); -#endif - /* Add an extra memory slot for testing dirty logging */ vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, guest_test_phys_mem, @@ -364,11 +444,8 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations, while (iteration < iterations) { /* Give the vcpu thread some time to dirty some pages */ usleep(interval * 1000); - kvm_vm_get_dirty_log(vm, TEST_MEM_SLOT_INDEX, bmap); -#ifdef USE_CLEAR_DIRTY_LOG - kvm_vm_clear_dirty_log(vm, TEST_MEM_SLOT_INDEX, bmap, 0, - host_num_pages); -#endif + log_mode_collect_dirty_pages(vm, TEST_MEM_SLOT_INDEX, + bmap, host_num_pages); vm_dirty_log_verify(bmap); iteration++; sync_global_to_guest(vm, iteration); @@ -413,6 +490,9 @@ static void help(char *name) TEST_HOST_LOOP_INTERVAL); printf(" -p: specify guest physical test memory offset\n" " Warning: a low offset can conflict with the loaded test code.\n"); + printf(" -M: specify the host logging mode " + "(default: log-dirty). Supported modes: \n\t"); + log_modes_dump(); printf(" -m: specify the guest mode ID to test " "(default: test all supported modes)\n" " This option may be used multiple times.\n" @@ -437,13 +517,6 @@ int main(int argc, char *argv[]) unsigned int host_ipa_limit; #endif -#ifdef USE_CLEAR_DIRTY_LOG - if (!kvm_check_cap(KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2)) { - fprintf(stderr, "KVM_CLEAR_DIRTY_LOG not available, skipping tests\n"); - exit(KSFT_SKIP); - } -#endif - #ifdef __x86_64__ vm_guest_mode_params_init(VM_MODE_PXXV48_4K, true, true); #endif @@ -463,7 +536,7 @@ int main(int argc, char *argv[]) vm_guest_mode_params_init(VM_MODE_P40V48_4K, true, true); #endif - while ((opt = getopt(argc, argv, "hi:I:p:m:")) != -1) { + while ((opt = getopt(argc, argv, "hi:I:p:m:M:")) != -1) { switch (opt) { case 'i': iterations = strtol(optarg, NULL, 10); @@ -485,6 +558,22 @@ int main(int argc, char *argv[]) "Guest mode ID %d too big", mode); vm_guest_mode_params[mode].enabled = true; break; + case 'M': + for (i = 0; i < LOG_MODE_NUM; i++) { + if (!strcmp(optarg, log_modes[i].name)) { + DEBUG("Setting log mode to: '%s'\n", + optarg); + host_log_mode = i; + break; + } + } + if (i == LOG_MODE_NUM) { + printf("Log mode '%s' is invalid. " + "Please choose from: ", optarg); + log_modes_dump(); + exit(-1); + } + break; case 'h': default: help(argv[0]); From patchwork Thu Jan 9 14:57:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325907 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A4009109A for ; Thu, 9 Jan 2020 14:58:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 824D92077B for ; Thu, 9 Jan 2020 14:58:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="OmDXlBvg" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730420AbgAIO6r (ORCPT ); Thu, 9 Jan 2020 09:58:47 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:22704 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732085AbgAIO6X (ORCPT ); Thu, 9 Jan 2020 09:58:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581901; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aF1BdxIc7NPI7Dsk+3ZH17n/EhvhXEPJcTNSoSkH9Wk=; b=OmDXlBvgH7rq8KYaqPFNdHAfEhiFkGrqhZDT2RPlzNI82hkUBVh0dJIsC/+qQm8o0XjiLz GjsK/NYNPOof+iSNZ2aLV9YCzzyWRas2ob0qMAvZFYcrQRcD4WlzRUyZczWwVVutkY/A5U O43gaiJRLWVbgcMZ9Tckwek/d2SZA9U= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-234-uU328G38NZWKiO9Mf87s7A-1; Thu, 09 Jan 2020 09:58:20 -0500 X-MC-Unique: uU328G38NZWKiO9Mf87s7A-1 Received: by mail-qv1-f69.google.com with SMTP id g6so4276815qvp.0 for ; Thu, 09 Jan 2020 06:58:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aF1BdxIc7NPI7Dsk+3ZH17n/EhvhXEPJcTNSoSkH9Wk=; b=e31XB1lrLgIAeI1zGKVgm76dPOIm8IbkfaFYj3FEKgsriAq3hsLI4UVTdwWCN0xFH/ 9iF96693WDHVMiQvIQljRULOd51M2hQLRDLP6ZQQBDEyCp08E/Ja5++MQ09BaHuz284q /WEOXi5GlzgQys++WI5HF+p5NjkJtUOde0jlzm9aWMmubZJXqIPqmacdgD2qk4o2NGwK kFyjLDIXah7f8H9D1XlCm+Ec5yTMSnHT7bwdSCUJ5JmPHQcj02PwtQqqSw7xEF4xsBaq oc/GDcOyNaDrabFkzswJcNkANtiRwlGQmTY9d7yLSMcBGUGEEIYlDFwaN+yLPqosMcvf 2YmQ== X-Gm-Message-State: APjAAAXhS7U7J+9+3MePywHB/oA4OUG/l8n4RJoR6HclFKoWLtEUddP2 YJ3M3zw2OnG5PCk6huvhbtf0iIh0PyuvNOljwpQZ6oes55G0iZR1ff+oJnOKaTRQFzqboEUPtdx u0D23607/XlU9 X-Received: by 2002:aed:36e5:: with SMTP id f92mr8461790qtb.354.1578581899551; Thu, 09 Jan 2020 06:58:19 -0800 (PST) X-Google-Smtp-Source: APXvYqyRwYVEXndV0U7BqbcKE99vl8mk+g1J4386LY+MgnHkwH4BOop64bg/+1s0dgEK/wCUhEN0lA== X-Received: by 2002:aed:36e5:: with SMTP id f92mr8461782qtb.354.1578581899350; Thu, 09 Jan 2020 06:58:19 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.58.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:58:18 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 18/21] KVM: selftests: Introduce after_vcpu_run hook for dirty log test Date: Thu, 9 Jan 2020 09:57:26 -0500 Message-Id: <20200109145729.32898-19-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Provide a hook for the checks after vcpu_run() completes. Preparation for the dirty ring test because we'll need to take care of another exit reason. Since at it, drop the pages_count because after all we have a better summary right now with statistics, and clean it up a bit. Signed-off-by: Peter Xu --- tools/testing/selftests/kvm/dirty_log_test.c | 39 ++++++++++++-------- 1 file changed, 23 insertions(+), 16 deletions(-) diff --git a/tools/testing/selftests/kvm/dirty_log_test.c b/tools/testing/selftests/kvm/dirty_log_test.c index a8ae8c0042a8..3542311f56ff 100644 --- a/tools/testing/selftests/kvm/dirty_log_test.c +++ b/tools/testing/selftests/kvm/dirty_log_test.c @@ -168,6 +168,15 @@ static void clear_log_collect_dirty_pages(struct kvm_vm *vm, int slot, kvm_vm_clear_dirty_log(vm, slot, bitmap, 0, num_pages); } +static void default_after_vcpu_run(struct kvm_vm *vm) +{ + struct kvm_run *run = vcpu_state(vm, VCPU_ID); + + TEST_ASSERT(get_ucall(vm, VCPU_ID, NULL) == UCALL_SYNC, + "Invalid guest sync status: exit_reason=%s\n", + exit_reason_str(run->exit_reason)); +} + struct log_mode { const char *name; /* Hook when the vm creation is done (before vcpu creation) */ @@ -175,16 +184,20 @@ struct log_mode { /* Hook to collect the dirty pages into the bitmap provided */ void (*collect_dirty_pages) (struct kvm_vm *vm, int slot, void *bitmap, uint32_t num_pages); + /* Hook to call when after each vcpu run */ + void (*after_vcpu_run)(struct kvm_vm *vm); } log_modes[LOG_MODE_NUM] = { { .name = "dirty-log", .create_vm_done = NULL, .collect_dirty_pages = dirty_log_collect_dirty_pages, + .after_vcpu_run = default_after_vcpu_run, }, { .name = "clear-log", .create_vm_done = clear_log_create_vm_done, .collect_dirty_pages = clear_log_collect_dirty_pages, + .after_vcpu_run = default_after_vcpu_run, }, }; @@ -224,6 +237,14 @@ static void log_mode_collect_dirty_pages(struct kvm_vm *vm, int slot, mode->collect_dirty_pages(vm, slot, bitmap, num_pages); } +static void log_mode_after_vcpu_run(struct kvm_vm *vm) +{ + struct log_mode *mode = &log_modes[host_log_mode]; + + if (mode->after_vcpu_run) + mode->after_vcpu_run(vm); +} + static void generate_random_array(uint64_t *guest_array, uint64_t size) { uint64_t i; @@ -237,31 +258,17 @@ static void *vcpu_worker(void *data) int ret; struct kvm_vm *vm = data; uint64_t *guest_array; - uint64_t pages_count = 0; - struct kvm_run *run; - - run = vcpu_state(vm, VCPU_ID); guest_array = addr_gva2hva(vm, (vm_vaddr_t)random_array); - generate_random_array(guest_array, TEST_PAGES_PER_LOOP); while (!READ_ONCE(host_quit)) { + generate_random_array(guest_array, TEST_PAGES_PER_LOOP); /* Let the guest dirty the random pages */ ret = _vcpu_run(vm, VCPU_ID); TEST_ASSERT(ret == 0, "vcpu_run failed: %d\n", ret); - if (get_ucall(vm, VCPU_ID, NULL) == UCALL_SYNC) { - pages_count += TEST_PAGES_PER_LOOP; - generate_random_array(guest_array, TEST_PAGES_PER_LOOP); - } else { - TEST_ASSERT(false, - "Invalid guest sync status: " - "exit_reason=%s\n", - exit_reason_str(run->exit_reason)); - } + log_mode_after_vcpu_run(vm); } - DEBUG("Dirtied %"PRIu64" pages\n", pages_count); - return NULL; } From patchwork Thu Jan 9 14:57:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325905 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1D7FD92A for ; Thu, 9 Jan 2020 14:58:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D21342072E for ; Thu, 9 Jan 2020 14:58:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="U1rMyyPI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730876AbgAIO6l (ORCPT ); Thu, 9 Jan 2020 09:58:41 -0500 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:52448 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732089AbgAIO6Y (ORCPT ); Thu, 9 Jan 2020 09:58:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581903; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HMDb7Z18c3Y1vpM+V/TCJrus+aRuAuf9YMwiY3aYrg8=; b=U1rMyyPIot0N8YfWG1DS9a7y1LaXmBK346xodJyyLH4Obfg2OG8W37EwKtm676DCHic7Nd Odvobp85BWqs63Zmlc2c0LYHT7IvmP3+1TCBJPREZd3GOrSjhsjzjfrUqgOsMA6PbPl+66 6Wdj21lD+AwqWIQEvAGgpxUp69xrN8k= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-397-PlsijYnjM7G8ecZjGcH56w-1; Thu, 09 Jan 2020 09:58:22 -0500 X-MC-Unique: PlsijYnjM7G8ecZjGcH56w-1 Received: by mail-qk1-f197.google.com with SMTP id 24so4262573qka.16 for ; Thu, 09 Jan 2020 06:58:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HMDb7Z18c3Y1vpM+V/TCJrus+aRuAuf9YMwiY3aYrg8=; b=AgAfrFf0z1nngjZ4zivSNK7+xwy9vFI7fNM4iSXUSH2zr+rCTAatidm3S0DmeYW6vT sJh72mXDppqyJecvkZO6Fih2dgZyolUqGufNzA3ndclz8aEGRfUi81ghjbJnG25NSz6B Ol+B5AR9uez5PHBVNw4iWp998Gy9/pRwgH14IPIwPvDm5LU7LT/OFsq5g24pvfm8J2Vy 0Xz/4fHj8CkfjtGrnzfX2U0+WW71FHrcirpiM2usCqiHKRe7yvv8dhxTvBZi2Jq4dV9P g2PzhhArPph5JIzZH9cHXPKMvEYNeNW1gEnjyQmHYpP7ouodpLxyXxku5dcb4GTJGDGH dLBQ== X-Gm-Message-State: APjAAAWBJS9TWZS6WPM4kM27gTpzXo/z5bQUkSxC9vm8YGFq6JTFLa3a 44lMENaheF4SFp9u5Xgvx0NVOztdmN7AAc/Uxgg+LkCuXazwvYlweKW+DjSS50VkAL/TDQmWA1j L+KetxRirHKk+ X-Received: by 2002:a0c:f703:: with SMTP id w3mr9245080qvn.6.1578581901302; Thu, 09 Jan 2020 06:58:21 -0800 (PST) X-Google-Smtp-Source: APXvYqwxAGptPj8oFZkWSPgJ5OF9Jgq+WrhZnsagmou2ZMuv4BQRDS3l2PQaAo5qvebR/6DDBPjNhw== X-Received: by 2002:a0c:f703:: with SMTP id w3mr9245037qvn.6.1578581900738; Thu, 09 Jan 2020 06:58:20 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.58.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:58:20 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 19/21] KVM: selftests: Add dirty ring buffer test Date: Thu, 9 Jan 2020 09:57:27 -0500 Message-Id: <20200109145729.32898-20-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add the initial dirty ring buffer test. The current test implements the userspace dirty ring collection, by only reaping the dirty ring when the ring is full. So it's still running synchronously like this: vcpu main thread 1. vcpu dirties pages 2. vcpu gets dirty ring full (userspace exit) 3. main thread waits until full (so hardware buffers flushed) 4. main thread collects 5. main thread continues vcpu 6. vcpu continues, goes back to 1 We can't directly collects dirty bits during vcpu execution because otherwise we can't guarantee the hardware dirty bits were flushed when we collect and we're very strict on the dirty bits so otherwise we can fail the future verify procedure. A follow up patch will make this test to support async just like the existing dirty log test, by adding a vcpu kick mechanism. Signed-off-by: Peter Xu --- tools/testing/selftests/kvm/dirty_log_test.c | 174 +++++++++++++++++- .../testing/selftests/kvm/include/kvm_util.h | 3 + tools/testing/selftests/kvm/lib/kvm_util.c | 64 +++++++ .../selftests/kvm/lib/kvm_util_internal.h | 3 + 4 files changed, 242 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kvm/dirty_log_test.c b/tools/testing/selftests/kvm/dirty_log_test.c index 3542311f56ff..6a551f285dea 100644 --- a/tools/testing/selftests/kvm/dirty_log_test.c +++ b/tools/testing/selftests/kvm/dirty_log_test.c @@ -12,8 +12,10 @@ #include #include #include +#include #include #include +#include #include "test_util.h" #include "kvm_util.h" @@ -57,6 +59,8 @@ # define test_and_clear_bit_le test_and_clear_bit #endif +#define TEST_DIRTY_RING_COUNT 1024 + /* * Guest/Host shared variables. Ensure addr_gva2hva() and/or * sync_global_to/from_guest() are used when accessing from @@ -128,6 +132,10 @@ static uint64_t host_dirty_count; static uint64_t host_clear_count; static uint64_t host_track_next_count; +/* Whether dirty ring reset is requested, or finished */ +static sem_t dirty_ring_vcpu_stop; +static sem_t dirty_ring_vcpu_cont; + enum log_mode_t { /* Only use KVM_GET_DIRTY_LOG for logging */ LOG_MODE_DIRTY_LOG = 0, @@ -135,6 +143,9 @@ enum log_mode_t { /* Use both KVM_[GET|CLEAR]_DIRTY_LOG for logging */ LOG_MODE_CLERA_LOG = 1, + /* Use dirty ring for logging */ + LOG_MODE_DIRTY_RING = 2, + LOG_MODE_NUM, }; @@ -177,6 +188,118 @@ static void default_after_vcpu_run(struct kvm_vm *vm) exit_reason_str(run->exit_reason)); } +static void dirty_ring_create_vm_done(struct kvm_vm *vm) +{ + /* + * Switch to dirty ring mode after VM creation but before any + * of the vcpu creation. + */ + vm_enable_dirty_ring(vm, TEST_DIRTY_RING_COUNT * + sizeof(struct kvm_dirty_gfn)); +} + +static uint32_t dirty_ring_collect_one(struct kvm_dirty_gfn *dirty_gfns, + struct kvm_dirty_ring_indices *indices, + int slot, void *bitmap, + uint32_t num_pages, int index) +{ + struct kvm_dirty_gfn *cur; + uint32_t avail, fetch, count = 0; + + /* + * We should keep it somewhere, but to be simple we read + * fetch_index too. + */ + fetch = READ_ONCE(indices->fetch_index); + avail = READ_ONCE(indices->avail_index); + + /* Make sure we read valid entries always */ + rmb(); + + DEBUG("ring %d: fetch: 0x%x, avail: 0x%x\n", index, fetch, avail); + + while (fetch != avail) { + cur = &dirty_gfns[fetch % TEST_DIRTY_RING_COUNT]; + TEST_ASSERT(cur->pad == 0, "Padding is non-zero: 0x%x", cur->pad); + TEST_ASSERT(cur->slot == slot, "Slot number didn't match: " + "%u != %u", cur->slot, slot); + TEST_ASSERT(cur->offset < num_pages, "Offset overflow: " + "0x%llx >= 0x%llx", cur->offset, num_pages); + DEBUG("fetch 0x%x offset 0x%llx\n", fetch, cur->offset); + set_bit(cur->offset, bitmap); + fetch++; + count++; + } + WRITE_ONCE(indices->fetch_index, fetch); + + return count; +} + +static void dirty_ring_collect_dirty_pages(struct kvm_vm *vm, int slot, + void *bitmap, uint32_t num_pages) +{ + /* We only have one vcpu */ + struct kvm_run *state = vcpu_state(vm, VCPU_ID); + uint32_t count = 0, cleared; + + /* + * Before fetching the dirty pages, we need a vmexit of the + * worker vcpu to make sure the hardware dirty buffers were + * flushed. This is not needed for dirty-log/clear-log tests + * because get dirty log will natually do so. + * + * For now we do it in the simple way - we simply wait until + * the vcpu uses up the soft dirty ring, then it'll always + * do a vmexit to make sure that PML buffers will be flushed. + * In real hypervisors, we probably need a vcpu kick or to + * stop the vcpus (before the final sync) to make sure we'll + * get all the existing dirty PFNs even cached in hardware. + */ + sem_wait(&dirty_ring_vcpu_stop); + + /* Only have one vcpu */ + count = dirty_ring_collect_one(vcpu_map_dirty_ring(vm, VCPU_ID), + &state->vcpu_ring_indices, + slot, bitmap, num_pages, VCPU_ID); + + cleared = kvm_vm_reset_dirty_ring(vm); + + /* Cleared pages should be the same as collected */ + TEST_ASSERT(cleared == count, "Reset dirty pages (%u) mismatch " + "with collected (%u)", cleared, count); + + DEBUG("Notifying vcpu to continue\n"); + sem_post(&dirty_ring_vcpu_cont); + + DEBUG("Iteration %ld collected %u pages\n", iteration, count); +} + +static void dirty_ring_after_vcpu_run(struct kvm_vm *vm) +{ + struct kvm_run *run = vcpu_state(vm, VCPU_ID); + + /* A ucall-sync or ring-full event is allowed */ + if (get_ucall(vm, VCPU_ID, NULL) == UCALL_SYNC) { + /* We should allow this to continue */ + ; + } else if (run->exit_reason == KVM_EXIT_DIRTY_RING_FULL) { + sem_post(&dirty_ring_vcpu_stop); + DEBUG("vcpu stops because dirty ring full...\n"); + sem_wait(&dirty_ring_vcpu_cont); + DEBUG("vcpu continues now.\n"); + } else { + TEST_ASSERT(false, "Invalid guest sync status: " + "exit_reason=%s\n", + exit_reason_str(run->exit_reason)); + } +} + +static void dirty_ring_before_vcpu_join(void) +{ + /* Kick another round of vcpu just to make sure it will quit */ + sem_post(&dirty_ring_vcpu_cont); +} + struct log_mode { const char *name; /* Hook when the vm creation is done (before vcpu creation) */ @@ -186,6 +309,7 @@ struct log_mode { void *bitmap, uint32_t num_pages); /* Hook to call when after each vcpu run */ void (*after_vcpu_run)(struct kvm_vm *vm); + void (*before_vcpu_join) (void); } log_modes[LOG_MODE_NUM] = { { .name = "dirty-log", @@ -199,6 +323,13 @@ struct log_mode { .collect_dirty_pages = clear_log_collect_dirty_pages, .after_vcpu_run = default_after_vcpu_run, }, + { + .name = "dirty-ring", + .create_vm_done = dirty_ring_create_vm_done, + .collect_dirty_pages = dirty_ring_collect_dirty_pages, + .before_vcpu_join = dirty_ring_before_vcpu_join, + .after_vcpu_run = dirty_ring_after_vcpu_run, + }, }; /* @@ -245,6 +376,14 @@ static void log_mode_after_vcpu_run(struct kvm_vm *vm) mode->after_vcpu_run(vm); } +static void log_mode_before_vcpu_join(void) +{ + struct log_mode *mode = &log_modes[host_log_mode]; + + if (mode->before_vcpu_join) + mode->before_vcpu_join(); +} + static void generate_random_array(uint64_t *guest_array, uint64_t size) { uint64_t i; @@ -292,14 +431,41 @@ static void vm_dirty_log_verify(unsigned long *bmap) } if (test_and_clear_bit_le(page, bmap)) { + bool matched; + host_dirty_count++; + /* * If the bit is set, the value written onto * the corresponding page should be either the * previous iteration number or the current one. + * + * (*value_ptr == iteration - 2) case is + * special only for dirty ring test where the + * page is the last page before a kvm dirty + * ring full userspace exit of the 2nd + * iteration, if without this we'll probably + * fail on the 4th iteration. Anyway, let's + * just loose the test case a little bit for + * all for simplicity. */ - TEST_ASSERT(*value_ptr == iteration || - *value_ptr == iteration - 1, + matched = (*value_ptr == iteration || + *value_ptr == iteration - 1 || + *value_ptr == iteration - 2); + + /* + * This is the common path for dirty ring + * where this page is exactly the last page + * touched before KVM_EXIT_DIRTY_RING_FULL. + * If it happens, we should expect it to be + * there for the next round. + */ + if (host_log_mode == LOG_MODE_DIRTY_RING && !matched) { + set_bit_le(page, host_bmap_track); + continue; + } + + TEST_ASSERT(matched, "Set page %"PRIu64" value %"PRIu64 " incorrect (iteration=%"PRIu64")", page, *value_ptr, iteration); @@ -460,6 +626,7 @@ static void run_test(enum vm_guest_mode mode, unsigned long iterations, /* Tell the vcpu thread to quit */ host_quit = true; + log_mode_before_vcpu_join(); pthread_join(vcpu_thread, NULL); DEBUG("Total bits checked: dirty (%"PRIu64"), clear (%"PRIu64"), " @@ -524,6 +691,9 @@ int main(int argc, char *argv[]) unsigned int host_ipa_limit; #endif + sem_init(&dirty_ring_vcpu_stop, 0, 0); + sem_init(&dirty_ring_vcpu_cont, 0, 0); + #ifdef __x86_64__ vm_guest_mode_params_init(VM_MODE_PXXV48_4K, true, true); #endif diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h index 29cccaf96baf..4b78a8d3e773 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -67,6 +67,7 @@ enum vm_mem_backing_src_type { int kvm_check_cap(long cap); int vm_enable_cap(struct kvm_vm *vm, struct kvm_enable_cap *cap); +void vm_enable_dirty_ring(struct kvm_vm *vm, uint32_t ring_size); struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm); struct kvm_vm *_vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm); @@ -76,6 +77,7 @@ void kvm_vm_release(struct kvm_vm *vmp); void kvm_vm_get_dirty_log(struct kvm_vm *vm, int slot, void *log); void kvm_vm_clear_dirty_log(struct kvm_vm *vm, int slot, void *log, uint64_t first_page, uint32_t num_pages); +uint32_t kvm_vm_reset_dirty_ring(struct kvm_vm *vm); int kvm_memcmp_hva_gva(void *hva, struct kvm_vm *vm, const vm_vaddr_t gva, size_t len); @@ -137,6 +139,7 @@ void vcpu_nested_state_get(struct kvm_vm *vm, uint32_t vcpuid, int vcpu_nested_state_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_nested_state *state, bool ignore_error); #endif +void *vcpu_map_dirty_ring(struct kvm_vm *vm, uint32_t vcpuid); const char *exit_reason_str(unsigned int exit_reason); diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c index 41cf45416060..81222e2f841e 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -85,6 +85,26 @@ int vm_enable_cap(struct kvm_vm *vm, struct kvm_enable_cap *cap) return ret; } +void vm_enable_dirty_ring(struct kvm_vm *vm, uint32_t ring_size) +{ + struct kvm_enable_cap cap = {}; + int ret; + + ret = kvm_check_cap(KVM_CAP_DIRTY_LOG_RING); + + TEST_ASSERT(ret >= 0, "KVM_CAP_DIRTY_LOG_RING"); + + if (ret == 0) { + fprintf(stderr, "KVM does not support dirty ring, skipping tests\n"); + exit(KSFT_SKIP); + } + + cap.cap = KVM_CAP_DIRTY_LOG_RING; + cap.args[0] = ring_size; + vm_enable_cap(vm, &cap); + vm->dirty_ring_size = ring_size; +} + static void vm_open(struct kvm_vm *vm, int perm) { vm->kvm_fd = open(KVM_DEV_PATH, perm); @@ -297,6 +317,11 @@ void kvm_vm_clear_dirty_log(struct kvm_vm *vm, int slot, void *log, strerror(-ret)); } +uint32_t kvm_vm_reset_dirty_ring(struct kvm_vm *vm) +{ + return ioctl(vm->fd, KVM_RESET_DIRTY_RINGS); +} + /* * Userspace Memory Region Find * @@ -408,6 +433,13 @@ static void vm_vcpu_rm(struct kvm_vm *vm, uint32_t vcpuid) struct vcpu *vcpu = vcpu_find(vm, vcpuid); int ret; + if (vcpu->dirty_gfns) { + ret = munmap(vcpu->dirty_gfns, vm->dirty_ring_size); + TEST_ASSERT(ret == 0, "munmap of VCPU dirty ring failed, " + "rc: %i errno: %i", ret, errno); + vcpu->dirty_gfns = NULL; + } + ret = munmap(vcpu->state, sizeof(*vcpu->state)); TEST_ASSERT(ret == 0, "munmap of VCPU fd failed, rc: %i " "errno: %i", ret, errno); @@ -1409,6 +1441,37 @@ int _vcpu_ioctl(struct kvm_vm *vm, uint32_t vcpuid, return ret; } +void *vcpu_map_dirty_ring(struct kvm_vm *vm, uint32_t vcpuid) +{ + struct vcpu *vcpu; + uint32_t size = vm->dirty_ring_size; + + TEST_ASSERT(size > 0, "Should enable dirty ring first"); + + vcpu = vcpu_find(vm, vcpuid); + + TEST_ASSERT(vcpu, "Cannot find vcpu %u", vcpuid); + + if (!vcpu->dirty_gfns) { + int prot = PROT_READ | PROT_WRITE; + void *addr; + + addr = mmap(NULL, size, prot, MAP_SHARED, vcpu->fd, + vm->page_size * KVM_DIRTY_LOG_PAGE_OFFSET); + TEST_ASSERT(addr == MAP_FAILED, "Dirty ring mapped writable"); + + prot = PROT_READ; + addr = mmap(NULL, size, prot, MAP_SHARED, vcpu->fd, + vm->page_size * KVM_DIRTY_LOG_PAGE_OFFSET); + TEST_ASSERT(addr != MAP_FAILED, "Dirty ring map failed"); + + vcpu->dirty_gfns = addr; + vcpu->dirty_gfns_count = size / sizeof(struct kvm_dirty_gfn); + } + + return vcpu->dirty_gfns; +} + /* * VM Ioctl * @@ -1503,6 +1566,7 @@ static struct exit_reason { {KVM_EXIT_INTERNAL_ERROR, "INTERNAL_ERROR"}, {KVM_EXIT_OSI, "OSI"}, {KVM_EXIT_PAPR_HCALL, "PAPR_HCALL"}, + {KVM_EXIT_DIRTY_RING_FULL, "DIRTY_RING_FULL"}, #ifdef KVM_EXIT_MEMORY_NOT_PRESENT {KVM_EXIT_MEMORY_NOT_PRESENT, "MEMORY_NOT_PRESENT"}, #endif diff --git a/tools/testing/selftests/kvm/lib/kvm_util_internal.h b/tools/testing/selftests/kvm/lib/kvm_util_internal.h index ac50c42750cf..87edcc6746a2 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util_internal.h +++ b/tools/testing/selftests/kvm/lib/kvm_util_internal.h @@ -39,6 +39,8 @@ struct vcpu { uint32_t id; int fd; struct kvm_run *state; + struct kvm_dirty_gfn *dirty_gfns; + uint32_t dirty_gfns_count; }; struct kvm_vm { @@ -61,6 +63,7 @@ struct kvm_vm { vm_paddr_t pgd; vm_vaddr_t gdt; vm_vaddr_t tss; + uint32_t dirty_ring_size; }; struct vcpu *vcpu_find(struct kvm_vm *vm, uint32_t vcpuid); From patchwork Thu Jan 9 14:57:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325901 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E0C1492A for ; Thu, 9 Jan 2020 14:58:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AAD1B2072A for ; Thu, 9 Jan 2020 14:58:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Y5m74uRm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732119AbgAIO62 (ORCPT ); Thu, 9 Jan 2020 09:58:28 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:23147 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732092AbgAIO6Z (ORCPT ); Thu, 9 Jan 2020 09:58:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581904; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fC+6Sh3GEblv8QYQZxVb8yxgjLMEdKyQTn5LrgCGUhc=; b=Y5m74uRm31TR/GdhSb6VsLikeTt5i21tDkGTI9m7G8YHDgFEfw1/i5xdWx7Lsx+bGp1Yqs iXjzScSSJ+GhVDLzzvc7mcm4j/IAYL17lDIB+n+pMLxhcTLTbIZOiTNehqHu2WH1M+JqDc R0GGUltYTpRAbRE24Y3MOpJ7/u8MmWQ= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-274-UKpgyCvLOQ-NWAWe7GRyQQ-1; Thu, 09 Jan 2020 09:58:23 -0500 X-MC-Unique: UKpgyCvLOQ-NWAWe7GRyQQ-1 Received: by mail-qv1-f72.google.com with SMTP id g6so4276881qvp.0 for ; Thu, 09 Jan 2020 06:58:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fC+6Sh3GEblv8QYQZxVb8yxgjLMEdKyQTn5LrgCGUhc=; b=clc2YhTMGF954oT85jAcOLQNWT6ch8nJmRgRqCF4M/QRhpeVYpURGdRSkdD/7CNLYu GR5hPgYH3bVloLvJT5+KhqhCk/GUnYGdMZ18o/pmwyS/BZqpZ0iJxX81UVfx3iP0WKH+ AlrLM+IgGOu5yNLiXpTXK02+yeuiWI6KbJFBRONTmBJRSDtRhXQJSt0TheUIkoAhcPiQ 7J8qdKAVv+DrC1vt36m984q2YxMr9I+oYEFHTR+uS0W3pn9nHD0VD82w+xVjtX+NyTOA LURVOTITfcZcsjn7CgyaeAQbwH6TiWd0zfgCzqXQn9tZv/NMLEbY0jDud4mlgt9AqkjH fDHA== X-Gm-Message-State: APjAAAV81sHtQ+oHDoYatxzZHW8DO/fcYW5v9tC6OMiH6x4v78lILTI1 Afd1eW17x6qW2Kxff4Dq1ICwrR+uKjB2ygmn+h6vfXzp6Q9umnHCR6Sx4kSNQS1OI1V8XqzMlg9 ypS7XuGfQy2j3 X-Received: by 2002:ac8:5448:: with SMTP id d8mr8213437qtq.205.1578581902194; Thu, 09 Jan 2020 06:58:22 -0800 (PST) X-Google-Smtp-Source: APXvYqyBQ589tcotZZtyB5tl1TvvysAeZEulRzXpyDWvFo9GTBU0Q9N0vpOgeIiNmNIf5Cz0DRzb1Q== X-Received: by 2002:ac8:5448:: with SMTP id d8mr8213408qtq.205.1578581901835; Thu, 09 Jan 2020 06:58:21 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.58.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:58:21 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 20/21] KVM: selftests: Let dirty_log_test async for dirty ring test Date: Thu, 9 Jan 2020 09:57:28 -0500 Message-Id: <20200109145729.32898-21-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Previously the dirty ring test was working in synchronous way, because only with a vmexit (with that it was the ring full event) we'll know the hardware dirty bits will be flushed to the dirty ring. With this patch we first introduced the vcpu kick mechanism by using SIGUSR1, meanwhile we can have a guarantee of vmexit and also the flushing of hardware dirty bits. With all these, we can keep the vcpu dirty work asynchronous of the whole collection procedure now. Still, we need to be very careful that we can only do it async if the vcpu is not reaching soft limit (no KVM_EXIT_DIRTY_RING_FULL). Otherwise we must collect the dirty bits before continuing the vcpu. Further increase the dirty ring size to current maximum to make sure we torture more on the no-ring-full case, which should be the major scenario when the hypervisors like QEMU would like to use this feature. Signed-off-by: Peter Xu --- tools/testing/selftests/kvm/dirty_log_test.c | 123 +++++++++++++----- .../testing/selftests/kvm/include/kvm_util.h | 1 + tools/testing/selftests/kvm/lib/kvm_util.c | 8 ++ 3 files changed, 103 insertions(+), 29 deletions(-) diff --git a/tools/testing/selftests/kvm/dirty_log_test.c b/tools/testing/selftests/kvm/dirty_log_test.c index 6a551f285dea..6da97e4a9408 100644 --- a/tools/testing/selftests/kvm/dirty_log_test.c +++ b/tools/testing/selftests/kvm/dirty_log_test.c @@ -13,6 +13,9 @@ #include #include #include +#include +#include +#include #include #include #include @@ -59,7 +62,9 @@ # define test_and_clear_bit_le test_and_clear_bit #endif -#define TEST_DIRTY_RING_COUNT 1024 +#define TEST_DIRTY_RING_COUNT 65536 + +#define SIG_IPI SIGUSR1 /* * Guest/Host shared variables. Ensure addr_gva2hva() and/or @@ -135,6 +140,12 @@ static uint64_t host_track_next_count; /* Whether dirty ring reset is requested, or finished */ static sem_t dirty_ring_vcpu_stop; static sem_t dirty_ring_vcpu_cont; +/* + * This is updated by the vcpu thread to tell the host whether it's a + * ring-full event. It should only be read until a sem_wait() of + * dirty_ring_vcpu_stop and before vcpu continues to run. + */ +static bool dirty_ring_vcpu_ring_full; enum log_mode_t { /* Only use KVM_GET_DIRTY_LOG for logging */ @@ -151,6 +162,33 @@ enum log_mode_t { /* Mode of logging. Default is LOG_MODE_DIRTY_LOG */ static enum log_mode_t host_log_mode; +pthread_t vcpu_thread; + +/* Only way to pass this to the signal handler */ +struct kvm_vm *current_vm; + +static void vcpu_sig_handler(int sig) +{ + TEST_ASSERT(sig == SIG_IPI, "unknown signal: %d", sig); +} + +static void vcpu_kick(void) +{ + pthread_kill(vcpu_thread, SIG_IPI); +} + +/* + * In our test we do signal tricks, let's use a better version of + * sem_wait to avoid signal interrupts + */ +static void sem_wait_until(sem_t *sem) +{ + int ret; + + do + ret = sem_wait(sem); + while (ret == -1 && errno == EINTR); +} static void clear_log_create_vm_done(struct kvm_vm *vm) { @@ -179,10 +217,13 @@ static void clear_log_collect_dirty_pages(struct kvm_vm *vm, int slot, kvm_vm_clear_dirty_log(vm, slot, bitmap, 0, num_pages); } -static void default_after_vcpu_run(struct kvm_vm *vm) +static void default_after_vcpu_run(struct kvm_vm *vm, int ret, int err) { struct kvm_run *run = vcpu_state(vm, VCPU_ID); + TEST_ASSERT(ret == 0 || (ret == -1 && err == EINTR), + "vcpu run failed: errno=%d", err); + TEST_ASSERT(get_ucall(vm, VCPU_ID, NULL) == UCALL_SYNC, "Invalid guest sync status: exit_reason=%s\n", exit_reason_str(run->exit_reason)); @@ -235,27 +276,37 @@ static uint32_t dirty_ring_collect_one(struct kvm_dirty_gfn *dirty_gfns, return count; } +static void dirty_ring_wait_vcpu(void) +{ + /* This makes sure that hardware PML cache flushed */ + vcpu_kick(); + sem_wait_until(&dirty_ring_vcpu_stop); +} + +static void dirty_ring_continue_vcpu(void) +{ + DEBUG("Notifying vcpu to continue\n"); + sem_post(&dirty_ring_vcpu_cont); +} + static void dirty_ring_collect_dirty_pages(struct kvm_vm *vm, int slot, void *bitmap, uint32_t num_pages) { /* We only have one vcpu */ struct kvm_run *state = vcpu_state(vm, VCPU_ID); uint32_t count = 0, cleared; + bool continued_vcpu = false; - /* - * Before fetching the dirty pages, we need a vmexit of the - * worker vcpu to make sure the hardware dirty buffers were - * flushed. This is not needed for dirty-log/clear-log tests - * because get dirty log will natually do so. - * - * For now we do it in the simple way - we simply wait until - * the vcpu uses up the soft dirty ring, then it'll always - * do a vmexit to make sure that PML buffers will be flushed. - * In real hypervisors, we probably need a vcpu kick or to - * stop the vcpus (before the final sync) to make sure we'll - * get all the existing dirty PFNs even cached in hardware. - */ - sem_wait(&dirty_ring_vcpu_stop); + dirty_ring_wait_vcpu(); + + if (!dirty_ring_vcpu_ring_full) { + /* + * This is not a ring-full event, it's safe to allow + * vcpu to continue + */ + dirty_ring_continue_vcpu(); + continued_vcpu = true; + } /* Only have one vcpu */ count = dirty_ring_collect_one(vcpu_map_dirty_ring(vm, VCPU_ID), @@ -268,13 +319,16 @@ static void dirty_ring_collect_dirty_pages(struct kvm_vm *vm, int slot, TEST_ASSERT(cleared == count, "Reset dirty pages (%u) mismatch " "with collected (%u)", cleared, count); - DEBUG("Notifying vcpu to continue\n"); - sem_post(&dirty_ring_vcpu_cont); + if (!continued_vcpu) { + TEST_ASSERT(dirty_ring_vcpu_ring_full, + "Didn't continue vcpu even without ring full"); + dirty_ring_continue_vcpu(); + } DEBUG("Iteration %ld collected %u pages\n", iteration, count); } -static void dirty_ring_after_vcpu_run(struct kvm_vm *vm) +static void dirty_ring_after_vcpu_run(struct kvm_vm *vm, int ret, int err) { struct kvm_run *run = vcpu_state(vm, VCPU_ID); @@ -282,10 +336,16 @@ static void dirty_ring_after_vcpu_run(struct kvm_vm *vm) if (get_ucall(vm, VCPU_ID, NULL) == UCALL_SYNC) { /* We should allow this to continue */ ; - } else if (run->exit_reason == KVM_EXIT_DIRTY_RING_FULL) { + } else if (run->exit_reason == KVM_EXIT_DIRTY_RING_FULL || + (ret == -1 && err == EINTR)) { + /* Update the flag first before pause */ + WRITE_ONCE(dirty_ring_vcpu_ring_full, + run->exit_reason == KVM_EXIT_DIRTY_RING_FULL); sem_post(&dirty_ring_vcpu_stop); - DEBUG("vcpu stops because dirty ring full...\n"); - sem_wait(&dirty_ring_vcpu_cont); + DEBUG("vcpu stops because %s...\n", + dirty_ring_vcpu_ring_full ? + "dirty ring is full" : "vcpu is kicked out"); + sem_wait_until(&dirty_ring_vcpu_cont); DEBUG("vcpu continues now.\n"); } else { TEST_ASSERT(false, "Invalid guest sync status: " @@ -308,7 +368,7 @@ struct log_mode { void (*collect_dirty_pages) (struct kvm_vm *vm, int slot, void *bitmap, uint32_t num_pages); /* Hook to call when after each vcpu run */ - void (*after_vcpu_run)(struct kvm_vm *vm); + void (*after_vcpu_run)(struct kvm_vm *vm, int ret, int err); void (*before_vcpu_join) (void); } log_modes[LOG_MODE_NUM] = { { @@ -368,12 +428,12 @@ static void log_mode_collect_dirty_pages(struct kvm_vm *vm, int slot, mode->collect_dirty_pages(vm, slot, bitmap, num_pages); } -static void log_mode_after_vcpu_run(struct kvm_vm *vm) +static void log_mode_after_vcpu_run(struct kvm_vm *vm, int ret, int err) { struct log_mode *mode = &log_modes[host_log_mode]; if (mode->after_vcpu_run) - mode->after_vcpu_run(vm); + mode->after_vcpu_run(vm, ret, err); } static void log_mode_before_vcpu_join(void) @@ -397,15 +457,21 @@ static void *vcpu_worker(void *data) int ret; struct kvm_vm *vm = data; uint64_t *guest_array; + struct sigaction sigact; + + current_vm = vm; + memset(&sigact, 0, sizeof(sigact)); + sigact.sa_handler = vcpu_sig_handler; + sigaction(SIG_IPI, &sigact, NULL); guest_array = addr_gva2hva(vm, (vm_vaddr_t)random_array); while (!READ_ONCE(host_quit)) { + /* Clear any existing kick signals */ generate_random_array(guest_array, TEST_PAGES_PER_LOOP); /* Let the guest dirty the random pages */ - ret = _vcpu_run(vm, VCPU_ID); - TEST_ASSERT(ret == 0, "vcpu_run failed: %d\n", ret); - log_mode_after_vcpu_run(vm); + ret = __vcpu_run(vm, VCPU_ID); + log_mode_after_vcpu_run(vm, ret, errno); } return NULL; @@ -528,7 +594,6 @@ static struct kvm_vm *create_vm(enum vm_guest_mode mode, uint32_t vcpuid, static void run_test(enum vm_guest_mode mode, unsigned long iterations, unsigned long interval, uint64_t phys_offset) { - pthread_t vcpu_thread; struct kvm_vm *vm; unsigned long *bmap; diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h index 4b78a8d3e773..e64fbfe6bbd5 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -115,6 +115,7 @@ vm_paddr_t addr_gva2gpa(struct kvm_vm *vm, vm_vaddr_t gva); struct kvm_run *vcpu_state(struct kvm_vm *vm, uint32_t vcpuid); void vcpu_run(struct kvm_vm *vm, uint32_t vcpuid); int _vcpu_run(struct kvm_vm *vm, uint32_t vcpuid); +int __vcpu_run(struct kvm_vm *vm, uint32_t vcpuid); void vcpu_run_complete_io(struct kvm_vm *vm, uint32_t vcpuid); void vcpu_set_mp_state(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_mp_state *mp_state); diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c index 81222e2f841e..12c83e2f3300 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -1187,6 +1187,14 @@ int _vcpu_run(struct kvm_vm *vm, uint32_t vcpuid) return rc; } +int __vcpu_run(struct kvm_vm *vm, uint32_t vcpuid) +{ + struct vcpu *vcpu = vcpu_find(vm, vcpuid); + + TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid); + return ioctl(vcpu->fd, KVM_RUN, NULL); +} + void vcpu_run_complete_io(struct kvm_vm *vm, uint32_t vcpuid) { struct vcpu *vcpu = vcpu_find(vm, vcpuid); From patchwork Thu Jan 9 14:57:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11325903 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DA3C892A for ; Thu, 9 Jan 2020 14:58:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B96142084D for ; Thu, 9 Jan 2020 14:58:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BxajOEqE" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732113AbgAIO61 (ORCPT ); Thu, 9 Jan 2020 09:58:27 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:50381 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732101AbgAIO6Z (ORCPT ); Thu, 9 Jan 2020 09:58:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1578581904; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N5/gZPNP/QItlxsKYnUSc3SUeJf7pxYxkCxufAe+9+I=; b=BxajOEqEIJ7ndmuFZpDL/qhxSK08cCt+pyK/7sg3V8B6ay4Qub3mnd6275EUZUmIv5UceU 0CA1Nz3PYMfrVVzauLLRwIDPyDezP13fXxq1Bb0N2Oys+UtbW2c6c0n7iv8hAcFiRNj4t+ XVaLJI315PpXjeJ5D3ECcgEJRsdyPVE= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-213-QR6wYwOCPbycDbna65x-6A-1; Thu, 09 Jan 2020 09:58:24 -0500 X-MC-Unique: QR6wYwOCPbycDbna65x-6A-1 Received: by mail-qk1-f197.google.com with SMTP id 12so4265221qkf.20 for ; Thu, 09 Jan 2020 06:58:23 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=N5/gZPNP/QItlxsKYnUSc3SUeJf7pxYxkCxufAe+9+I=; b=Et16tJPi5BIdEAOOPfuZeNfqDL/biAQotlXQi0PQxDVrrmEirC42Yj8qHheeD17hox Q/ht8LTql+og+OhrNKvvdk51G4RZ15a4uDME+n5N6pSrGmeQseVot9TgpDK55mdDdw4e oa/U+64W5OW/uxtu8NX8yj3j4YSzW6ZFKgDqYk1uzBiYHyw4U5mrGVP5yl5NrJG3lI99 xESLBjxCw2Io0u9TwBNbVJ0lTWHDk29LM7Ju/zWdzC0AIDeSeigfjLXKk3ZxfbR5gLSi NfFQzUtqyT6LxUb7ZYeGAjCIloT10zDN1hOeAc5mUBAubvnvHf1x45GCa8Y/BPcsXWEo p3mQ== X-Gm-Message-State: APjAAAV/oAIcIFEnIUD0Me6TNmDbBVdVyJSe6qyHNPLlWCdAlhCoJrmB 1JUq3iWsFIYkBa+vCwvn7PnX/JFqIVY3CX0nhkiXdYP9Sb60K99tQEns35HHJt9BJ4f5d74Gr2d VUJh2aFzWumqC X-Received: by 2002:a05:620a:1108:: with SMTP id o8mr9361845qkk.118.1578581903456; Thu, 09 Jan 2020 06:58:23 -0800 (PST) X-Google-Smtp-Source: APXvYqwbk+PEzju7o1lLOeEMcQhDn/PrsxAenkqs3X56jNWYL+hX9tnG49Zv7cb0rnQ4BjnnJdD5RQ== X-Received: by 2002:a05:620a:1108:: with SMTP id o8mr9361819qkk.118.1578581903204; Thu, 09 Jan 2020 06:58:23 -0800 (PST) Received: from xz-x1.yyz.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id q2sm3124179qkm.5.2020.01.09.06.58.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2020 06:58:22 -0800 (PST) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Christophe de Dinechin , "Michael S . Tsirkin" , Paolo Bonzini , Sean Christopherson , Yan Zhao , Alex Williamson , Jason Wang , Kevin Kevin , Vitaly Kuznetsov , peterx@redhat.com, "Dr . David Alan Gilbert" Subject: [PATCH v3 21/21] KVM: selftests: Add "-c" parameter to dirty log test Date: Thu, 9 Jan 2020 09:57:29 -0500 Message-Id: <20200109145729.32898-22-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200109145729.32898-1-peterx@redhat.com> References: <20200109145729.32898-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org It's only used to override the existing dirty ring size/count. If with a bigger ring count, we test async of dirty ring. If with a smaller ring count, we test ring full code path. Async is default. It has no use for non-dirty-ring tests. Signed-off-by: Peter Xu --- tools/testing/selftests/kvm/dirty_log_test.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/kvm/dirty_log_test.c b/tools/testing/selftests/kvm/dirty_log_test.c index 6da97e4a9408..fb6c33dbaf35 100644 --- a/tools/testing/selftests/kvm/dirty_log_test.c +++ b/tools/testing/selftests/kvm/dirty_log_test.c @@ -163,6 +163,7 @@ enum log_mode_t { /* Mode of logging. Default is LOG_MODE_DIRTY_LOG */ static enum log_mode_t host_log_mode; pthread_t vcpu_thread; +static uint32_t test_dirty_ring_count = TEST_DIRTY_RING_COUNT; /* Only way to pass this to the signal handler */ struct kvm_vm *current_vm; @@ -235,7 +236,7 @@ static void dirty_ring_create_vm_done(struct kvm_vm *vm) * Switch to dirty ring mode after VM creation but before any * of the vcpu creation. */ - vm_enable_dirty_ring(vm, TEST_DIRTY_RING_COUNT * + vm_enable_dirty_ring(vm, test_dirty_ring_count * sizeof(struct kvm_dirty_gfn)); } @@ -260,7 +261,7 @@ static uint32_t dirty_ring_collect_one(struct kvm_dirty_gfn *dirty_gfns, DEBUG("ring %d: fetch: 0x%x, avail: 0x%x\n", index, fetch, avail); while (fetch != avail) { - cur = &dirty_gfns[fetch % TEST_DIRTY_RING_COUNT]; + cur = &dirty_gfns[fetch % test_dirty_ring_count]; TEST_ASSERT(cur->pad == 0, "Padding is non-zero: 0x%x", cur->pad); TEST_ASSERT(cur->slot == slot, "Slot number didn't match: " "%u != %u", cur->slot, slot); @@ -723,6 +724,9 @@ static void help(char *name) printf("usage: %s [-h] [-i iterations] [-I interval] " "[-p offset] [-m mode]\n", name); puts(""); + printf(" -c: specify dirty ring size, in number of entries\n"); + printf(" (only useful for dirty-ring test; default: %"PRIu32")\n", + TEST_DIRTY_RING_COUNT); printf(" -i: specify iteration counts (default: %"PRIu64")\n", TEST_HOST_LOOP_N); printf(" -I: specify interval in ms (default: %"PRIu64" ms)\n", @@ -778,8 +782,11 @@ int main(int argc, char *argv[]) vm_guest_mode_params_init(VM_MODE_P40V48_4K, true, true); #endif - while ((opt = getopt(argc, argv, "hi:I:p:m:M:")) != -1) { + while ((opt = getopt(argc, argv, "c:hi:I:p:m:M:")) != -1) { switch (opt) { + case 'c': + test_dirty_ring_count = strtol(optarg, NULL, 10); + break; case 'i': iterations = strtol(optarg, NULL, 10); break;