From patchwork Fri Dec 20 21:03:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11306301 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 24B3E14B7 for ; Fri, 20 Dec 2019 21:03:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0382621655 for ; Fri, 20 Dec 2019 21:03:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hov/puDt" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727577AbfLTVDg (ORCPT ); Fri, 20 Dec 2019 16:03:36 -0500 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:50966 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727730AbfLTVDe (ORCPT ); Fri, 20 Dec 2019 16:03:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576875813; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5u+NdTdcxJqnd4CDcK36kvrGYVN4B69rZJAB5gjRDOQ=; b=hov/puDtQHvNvWohlhaKQ3YJmaHFG7oHz1Chhy1SMre/gyIWSWLxg3wOPvbXPuTe3MZPDF hAbqcvanMcrcpMKAbS+QmRoQtqj3MKB822n0FnpjBPcACXsqpsyUrq4Wv9/3QV5eMaWtQ7 BDz5C8pnlcYFgtaECJvGpGgp7aoPj44= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-243-wb3gFqIjPBWklJFfeB_KtQ-1; Fri, 20 Dec 2019 16:03:31 -0500 X-MC-Unique: wb3gFqIjPBWklJFfeB_KtQ-1 Received: by mail-qv1-f72.google.com with SMTP id di5so1662368qvb.3 for ; Fri, 20 Dec 2019 13:03:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5u+NdTdcxJqnd4CDcK36kvrGYVN4B69rZJAB5gjRDOQ=; b=RqHdWa3C67jDbAxE5qBKvQ2HeotB80h4OiIy6rDdMxq9AtJKq3sdvyN8bUzcqd+UkB CniOKVSC4j49ejY+jBDADopXzStE7zsbC2hgMzKZZf/F3nVO/I/KNyeKHysfay1nY0KD evXTs7uWn9CsFoa4lWZu4jtCMlOzu7O/B1BX5uUjWgfROW6HcLzp7RIbbymh6+/xZ4As cPffHFiH+vLIoWmiDHH2hz32PPiV7iGLhl1ueLrjrUS1EjPn1nNQl5y7JzRPfll+jdr2 rw+ceXQe0YGW3WCBaX7LJbuIfMFCo/ausJRL34JQWpK1+ty+v5abBtJJmD35WXwQrgN0 pONw== X-Gm-Message-State: APjAAAWu1sD7HxCbmqzWOqF9vjHjxEBUBZw0omUWU0bVMnUBSJ/ljKJw XZxMflC6NQM2lDZIgRSqItp66DnUUCdYW0Owd851NYlywtOKuHfFRSVi8CRoq9vMJesQtdPTFDn tXk1GXe4I4w5s X-Received: by 2002:ac8:4456:: with SMTP id m22mr13436033qtn.362.1576875810875; Fri, 20 Dec 2019 13:03:30 -0800 (PST) X-Google-Smtp-Source: APXvYqwjedy9vJO9QU3awwLVGvmg81TcgWXkz+6waubY5bhmdyM90H3Ldeop7O/CBvSPdtrUYJzuww== X-Received: by 2002:ac8:4456:: with SMTP id m22mr13436025qtn.362.1576875810722; Fri, 20 Dec 2019 13:03:30 -0800 (PST) Received: from xz-x1.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id a9sm3061018qtb.36.2019.12.20.13.03.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2019 13:03:29 -0800 (PST) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterx@redhat.com, Sean Christopherson , "Michael S . Tsirkin" , Vitaly Kuznetsov , Paolo Bonzini , Christophe de Dinechin , "Dr . David Alan Gilbert" , Jason Wang Subject: [PATCH v2 01/17] KVM: Remove kvm_read_guest_atomic() Date: Fri, 20 Dec 2019 16:03:10 -0500 Message-Id: <20191220210326.49949-2-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20191220210326.49949-1-peterx@redhat.com> References: <20191220210326.49949-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Remove kvm_read_guest_atomic() because it's not used anywhere. Signed-off-by: Peter Xu --- include/linux/kvm_host.h | 2 -- virt/kvm/kvm_main.c | 11 ----------- 2 files changed, 13 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index d41c521a39da..2ea1ea79befd 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -730,8 +730,6 @@ void kvm_get_pfn(kvm_pfn_t pfn); int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset, int len); -int kvm_read_guest_atomic(struct kvm *kvm, gpa_t gpa, void *data, - unsigned long len); int kvm_read_guest(struct kvm *kvm, gpa_t gpa, void *data, unsigned long len); int kvm_read_guest_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, void *data, unsigned long len); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 13efc291b1c7..7ee28af9eb48 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2039,17 +2039,6 @@ static int __kvm_read_guest_atomic(struct kvm_memory_slot *slot, gfn_t gfn, return 0; } -int kvm_read_guest_atomic(struct kvm *kvm, gpa_t gpa, void *data, - unsigned long len) -{ - gfn_t gfn = gpa >> PAGE_SHIFT; - struct kvm_memory_slot *slot = gfn_to_memslot(kvm, gfn); - int offset = offset_in_page(gpa); - - return __kvm_read_guest_atomic(slot, gfn, data, offset, len); -} -EXPORT_SYMBOL_GPL(kvm_read_guest_atomic); - int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu, gpa_t gpa, void *data, unsigned long len) { From patchwork Fri Dec 20 21:03:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11306303 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E8AAC14B7 for ; Fri, 20 Dec 2019 21:03:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C545C218AC for ; Fri, 20 Dec 2019 21:03:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="g338PQH4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727747AbfLTVDi (ORCPT ); Fri, 20 Dec 2019 16:03:38 -0500 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:26611 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727749AbfLTVDg (ORCPT ); Fri, 20 Dec 2019 16:03:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576875814; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m4Ydwgp5PpMnSage4lUP1t9xcFgD9PS/bCLMno8IgRs=; b=g338PQH4L5FEQFySm2VjtVGXXjdOgtOAPLXB9UEXosY+8CPBmhXS/9Jugq1+hRNe+xwYFX 94TIHhAoU79ZtJqC36YX7ZzYrds+7WRxpZFkqRRa7Zd2NfFKT9b9aNaDZ7UKKUsCDUdrLe idIRP1YDQoAYdkEKhYa1BYCPkUZ8OLU= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-216-sO2YBFtBOlSzo0I8d83xYw-1; Fri, 20 Dec 2019 16:03:33 -0500 X-MC-Unique: sO2YBFtBOlSzo0I8d83xYw-1 Received: by mail-qv1-f71.google.com with SMTP id k2so6692186qvu.22 for ; Fri, 20 Dec 2019 13:03:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=m4Ydwgp5PpMnSage4lUP1t9xcFgD9PS/bCLMno8IgRs=; b=Nu+mmFO/zBqOJEmPg1b+511yC1Z9L9WTwfG5b2tUDJPaJRiyHmHnKniYgqG1kLAnjE Q06tdNlasG8bif5eU9WouUiOEhBFWOSDDvlnXeIEdtX68guw/RIbY/VYHkX96N3fkxYb 5Zp5Qt9tZU2rjjI4BBT87ha/AKKjBmnZ7aZb3SqJrmMci3TaPvO/DDLP7MpGmXMLxQdJ Cqu83zlgZW9w4v8IPmL/4cfXIAnaHLe3Tk6BojJDa9fu0S5fmz6A/HyUTMa2lPPgISLL 7OnLdHVfUar3OoLgNvxC3O0CLFdadry2Uu8AFEPkBDpfRd1U72ttWmUGDX6gkGpveG7j CoKw== X-Gm-Message-State: APjAAAWzdwO8xR6hr9L5KcwQyLu0tBOqkVNYzLHlFkf1awuxUa9QzfWS w87LuTF0oAYzUSz1ZzrmR4v9LFRMKY0eucKebKywZtGGt4JE9paeGpxTR64nFe66ZCZFkEyx9xT 4aB21pOKH/IoI X-Received: by 2002:a05:620a:1324:: with SMTP id p4mr15563894qkj.497.1576875812886; Fri, 20 Dec 2019 13:03:32 -0800 (PST) X-Google-Smtp-Source: APXvYqyzpJoDwbumEgXM2rd/MSIPuBvYBc/66AQA/wq59Gx92NiDKpzaG+U5dMYCgnpLzFCTUZBrpA== X-Received: by 2002:a05:620a:1324:: with SMTP id p4mr15563877qkj.497.1576875812682; Fri, 20 Dec 2019 13:03:32 -0800 (PST) Received: from xz-x1.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id a9sm3061018qtb.36.2019.12.20.13.03.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2019 13:03:31 -0800 (PST) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterx@redhat.com, Sean Christopherson , "Michael S . Tsirkin" , Vitaly Kuznetsov , Paolo Bonzini , Christophe de Dinechin , "Dr . David Alan Gilbert" , Jason Wang Subject: [PATCH v2 02/17] KVM: X86: Change parameter for fast_page_fault tracepoint Date: Fri, 20 Dec 2019 16:03:11 -0500 Message-Id: <20191220210326.49949-3-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20191220210326.49949-1-peterx@redhat.com> References: <20191220210326.49949-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org It would be clearer to dump the return value to know easily on whether did we go through the fast path for handling current page fault. Remove the old two last parameters because after all the old/new sptes were dumped in the same line. Signed-off-by: Peter Xu --- arch/x86/kvm/mmutrace.h | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/mmutrace.h b/arch/x86/kvm/mmutrace.h index 7ca8831c7d1a..09bdc5c91650 100644 --- a/arch/x86/kvm/mmutrace.h +++ b/arch/x86/kvm/mmutrace.h @@ -244,9 +244,6 @@ TRACE_EVENT( __entry->access) ); -#define __spte_satisfied(__spte) \ - (__entry->retry && is_writable_pte(__entry->__spte)) - TRACE_EVENT( fast_page_fault, TP_PROTO(struct kvm_vcpu *vcpu, gva_t gva, u32 error_code, @@ -274,12 +271,10 @@ TRACE_EVENT( ), TP_printk("vcpu %d gva %lx error_code %s sptep %p old %#llx" - " new %llx spurious %d fixed %d", __entry->vcpu_id, + " new %llx ret %d", __entry->vcpu_id, __entry->gva, __print_flags(__entry->error_code, "|", kvm_mmu_trace_pferr_flags), __entry->sptep, - __entry->old_spte, __entry->new_spte, - __spte_satisfied(old_spte), __spte_satisfied(new_spte) - ) + __entry->old_spte, __entry->new_spte, __entry->retry) ); TRACE_EVENT( From patchwork Fri Dec 20 21:03:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11306317 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B968114B7 for ; Fri, 20 Dec 2019 21:04:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 84F8A2146E for ; Fri, 20 Dec 2019 21:04:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hDjF5nu/" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727584AbfLTVEQ (ORCPT ); Fri, 20 Dec 2019 16:04:16 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:26523 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727753AbfLTVDh (ORCPT ); Fri, 20 Dec 2019 16:03:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576875816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xQjPBgffYJmUdXwhQa7UpTs/UC7BUPavY9NeRZmBhog=; b=hDjF5nu/o5yxCYSISWHh6JSP9y1Nx9HcybkbUJQsJegQiWL6sL7nlLI1Uer973OiRbIYwV /7OhGsNudSzMeQ1p40qk5E4+7tJpTtk13e1Ue5MQTc3n+N5EurUuuFDtlQpqzqV9iaYukU zLe8vg4EcFne1NBZZ76IEd3NjPSSWXc= Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-351-rb_SPIdbOAK0o2KJzcnRXw-1; Fri, 20 Dec 2019 16:03:35 -0500 X-MC-Unique: rb_SPIdbOAK0o2KJzcnRXw-1 Received: by mail-qt1-f199.google.com with SMTP id e1so3615227qto.5 for ; Fri, 20 Dec 2019 13:03:34 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xQjPBgffYJmUdXwhQa7UpTs/UC7BUPavY9NeRZmBhog=; b=fHIHxzXsPOGg679dxTENfvJHHcMqhjOHPwqvkbRpJ6kp37usD0hel0l5WlIwKq3j1G 9SDe8uXOh+mbHXhDr1IDFYjTfvADQF4SlIqSxvX/ysa6s7QXgvzlIQuAL+Yw/x2myUlG huje90waZpPVNE4FM8Cr/4XZmr389I9HLgRyFoQd8L/shWLQqJRWQRGhgXrZTAT33NHW PeQqsqXp1EIVK715YHL10hTDDESQeA1JXKtQ/O8fUOAty9Ad8yN701Lbu0UieePfn2Yc HhHBw0qTJIIGmintCuQbXS8d+wSehrxaCWE7o1D6o8gJvObaSHucQ7323eEo1yk5rS2R FlsQ== X-Gm-Message-State: APjAAAVHH0e8BgzSARHrKZS4/DniV9/O0VEo6oHuxG/NVhNbqCeraapP y/7xs0sIo+vvFm38bZ6oZI+S28a81RFoShfa8N1WB0P5benFAjHCVtUhqIEM23T77uMFz/+KBOE 1/7bKQuGXA46J X-Received: by 2002:a05:6214:1242:: with SMTP id q2mr14553006qvv.178.1576875814460; Fri, 20 Dec 2019 13:03:34 -0800 (PST) X-Google-Smtp-Source: APXvYqy6yp74ZU9GmcVHYyPvJDbmjLbviWCIFhfw5uOJc4g7munlnIsjnUNTt0ayFYzkEn6odRJkYQ== X-Received: by 2002:a05:6214:1242:: with SMTP id q2mr14552981qvv.178.1576875814167; Fri, 20 Dec 2019 13:03:34 -0800 (PST) Received: from xz-x1.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id a9sm3061018qtb.36.2019.12.20.13.03.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2019 13:03:33 -0800 (PST) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterx@redhat.com, Sean Christopherson , "Michael S . Tsirkin" , Vitaly Kuznetsov , Paolo Bonzini , Christophe de Dinechin , "Dr . David Alan Gilbert" , Jason Wang Subject: [PATCH v2 03/17] KVM: X86: Don't track dirty for KVM_SET_[TSS_ADDR|IDENTITY_MAP_ADDR] Date: Fri, 20 Dec 2019 16:03:12 -0500 Message-Id: <20191220210326.49949-4-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20191220210326.49949-1-peterx@redhat.com> References: <20191220210326.49949-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Originally, we have three code paths that can dirty a page without vcpu context for X86: - init_rmode_identity_map - init_rmode_tss - kvmgt_rw_gpa init_rmode_identity_map and init_rmode_tss will be setup on destination VM no matter what (and the guest cannot even see them), so it does not make sense to track them at all. To do this, a new parameter is added to kvm_[write|clear]_guest_page() to show whether we would like to track dirty bits for the operations. With that, pass in "false" to this new parameter for any guest memory write of the ioctls (KVM_SET_TSS_ADDR, KVM_SET_IDENTITY_MAP_ADDR). Signed-off-by: Peter Xu --- arch/x86/kvm/vmx/vmx.c | 18 ++++++++++-------- include/linux/kvm_host.h | 5 +++-- virt/kvm/kvm_main.c | 25 ++++++++++++++++--------- 3 files changed, 29 insertions(+), 19 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 04a8212704c1..1ff5a428f489 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -3452,24 +3452,24 @@ static int init_rmode_tss(struct kvm *kvm) idx = srcu_read_lock(&kvm->srcu); fn = to_kvm_vmx(kvm)->tss_addr >> PAGE_SHIFT; - r = kvm_clear_guest_page(kvm, fn, 0, PAGE_SIZE); + r = kvm_clear_guest_page(kvm, fn, 0, PAGE_SIZE, false); if (r < 0) goto out; data = TSS_BASE_SIZE + TSS_REDIRECTION_SIZE; r = kvm_write_guest_page(kvm, fn++, &data, - TSS_IOPB_BASE_OFFSET, sizeof(u16)); + TSS_IOPB_BASE_OFFSET, sizeof(u16), false); if (r < 0) goto out; - r = kvm_clear_guest_page(kvm, fn++, 0, PAGE_SIZE); + r = kvm_clear_guest_page(kvm, fn++, 0, PAGE_SIZE, false); if (r < 0) goto out; - r = kvm_clear_guest_page(kvm, fn, 0, PAGE_SIZE); + r = kvm_clear_guest_page(kvm, fn, 0, PAGE_SIZE, false); if (r < 0) goto out; data = ~0; r = kvm_write_guest_page(kvm, fn, &data, RMODE_TSS_SIZE - 2 * PAGE_SIZE - 1, - sizeof(u8)); + sizeof(u8), false); out: srcu_read_unlock(&kvm->srcu, idx); return r; @@ -3498,7 +3498,7 @@ static int init_rmode_identity_map(struct kvm *kvm) goto out2; idx = srcu_read_lock(&kvm->srcu); - r = kvm_clear_guest_page(kvm, identity_map_pfn, 0, PAGE_SIZE); + r = kvm_clear_guest_page(kvm, identity_map_pfn, 0, PAGE_SIZE, false); if (r < 0) goto out; /* Set up identity-mapping pagetable for EPT in real mode */ @@ -3506,7 +3506,8 @@ static int init_rmode_identity_map(struct kvm *kvm) tmp = (i << 22) + (_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | _PAGE_ACCESSED | _PAGE_DIRTY | _PAGE_PSE); r = kvm_write_guest_page(kvm, identity_map_pfn, - &tmp, i * sizeof(tmp), sizeof(tmp)); + &tmp, i * sizeof(tmp), + sizeof(tmp), false); if (r < 0) goto out; } @@ -7265,7 +7266,8 @@ static int vmx_write_pml_buffer(struct kvm_vcpu *vcpu) dst = vmcs12->pml_address + sizeof(u64) * vmcs12->guest_pml_index; if (kvm_write_guest_page(vcpu->kvm, gpa_to_gfn(dst), &gpa, - offset_in_page(dst), sizeof(gpa))) + offset_in_page(dst), sizeof(gpa), + false)) return 0; vmcs12->guest_pml_index--; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 2ea1ea79befd..4e34cf97ca90 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -734,7 +734,7 @@ int kvm_read_guest(struct kvm *kvm, gpa_t gpa, void *data, unsigned long len); int kvm_read_guest_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, void *data, unsigned long len); int kvm_write_guest_page(struct kvm *kvm, gfn_t gfn, const void *data, - int offset, int len); + int offset, int len, bool track_dirty); int kvm_write_guest(struct kvm *kvm, gpa_t gpa, const void *data, unsigned long len); int kvm_write_guest_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, @@ -744,7 +744,8 @@ int kvm_write_guest_offset_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, unsigned long len); int kvm_gfn_to_hva_cache_init(struct kvm *kvm, struct gfn_to_hva_cache *ghc, gpa_t gpa, unsigned long len); -int kvm_clear_guest_page(struct kvm *kvm, gfn_t gfn, int offset, int len); +int kvm_clear_guest_page(struct kvm *kvm, gfn_t gfn, int offset, int len, + bool track_dirty); int kvm_clear_guest(struct kvm *kvm, gpa_t gpa, unsigned long len); struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn); bool kvm_is_visible_gfn(struct kvm *kvm, gfn_t gfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 7ee28af9eb48..b1047173d78e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2051,7 +2051,8 @@ int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu, gpa_t gpa, EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_atomic); static int __kvm_write_guest_page(struct kvm_memory_slot *memslot, gfn_t gfn, - const void *data, int offset, int len) + const void *data, int offset, int len, + bool track_dirty) { int r; unsigned long addr; @@ -2062,16 +2063,19 @@ static int __kvm_write_guest_page(struct kvm_memory_slot *memslot, gfn_t gfn, r = __copy_to_user((void __user *)addr + offset, data, len); if (r) return -EFAULT; - mark_page_dirty_in_slot(memslot, gfn); + if (track_dirty) + mark_page_dirty_in_slot(memslot, gfn); return 0; } int kvm_write_guest_page(struct kvm *kvm, gfn_t gfn, - const void *data, int offset, int len) + const void *data, int offset, int len, + bool track_dirty) { struct kvm_memory_slot *slot = gfn_to_memslot(kvm, gfn); - return __kvm_write_guest_page(slot, gfn, data, offset, len); + return __kvm_write_guest_page(slot, gfn, data, offset, len, + track_dirty); } EXPORT_SYMBOL_GPL(kvm_write_guest_page); @@ -2080,7 +2084,8 @@ int kvm_vcpu_write_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, { struct kvm_memory_slot *slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn); - return __kvm_write_guest_page(slot, gfn, data, offset, len); + return __kvm_write_guest_page(slot, gfn, data, offset, + len, true); } EXPORT_SYMBOL_GPL(kvm_vcpu_write_guest_page); @@ -2093,7 +2098,7 @@ int kvm_write_guest(struct kvm *kvm, gpa_t gpa, const void *data, int ret; while ((seg = next_segment(len, offset)) != 0) { - ret = kvm_write_guest_page(kvm, gfn, data, offset, seg); + ret = kvm_write_guest_page(kvm, gfn, data, offset, seg, true); if (ret < 0) return ret; offset = 0; @@ -2232,11 +2237,13 @@ int kvm_read_guest_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, } EXPORT_SYMBOL_GPL(kvm_read_guest_cached); -int kvm_clear_guest_page(struct kvm *kvm, gfn_t gfn, int offset, int len) +int kvm_clear_guest_page(struct kvm *kvm, gfn_t gfn, int offset, int len, + bool track_dirty) { const void *zero_page = (const void *) __va(page_to_phys(ZERO_PAGE(0))); - return kvm_write_guest_page(kvm, gfn, zero_page, offset, len); + return kvm_write_guest_page(kvm, gfn, zero_page, offset, len, + track_dirty); } EXPORT_SYMBOL_GPL(kvm_clear_guest_page); @@ -2248,7 +2255,7 @@ int kvm_clear_guest(struct kvm *kvm, gpa_t gpa, unsigned long len) int ret; while ((seg = next_segment(len, offset)) != 0) { - ret = kvm_clear_guest_page(kvm, gfn, offset, seg); + ret = kvm_clear_guest_page(kvm, gfn, offset, seg, true); if (ret < 0) return ret; offset = 0; From patchwork Fri Dec 20 21:03:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11306305 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0D715139A for ; Fri, 20 Dec 2019 21:03:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E082921655 for ; Fri, 20 Dec 2019 21:03:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="JHYlASTk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727754AbfLTVDk (ORCPT ); Fri, 20 Dec 2019 16:03:40 -0500 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:21344 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727764AbfLTVDj (ORCPT ); Fri, 20 Dec 2019 16:03:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576875818; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5JYh7n3fhb27PXs/bYUs6lzlcg1ya8dgcofIlxRo1Jg=; b=JHYlASTk2n2IpYAqnVsa8zL6HxT+rixFPxOVx7W3N0A6vXsWPXj/gMdLPl/D+dYjgyMLlD Mxjda9Yq29kC8UYJgwNDL1zRiYr354/Jk8G9UWK+S8jtSjqnMDg6JqUVnZFKuQtOton2BB zeD2V9Lafngk+3gzP1QXWCO6bM+FEwA= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-235-lte1sWzTPa-FCkZeul_B_w-1; Fri, 20 Dec 2019 16:03:37 -0500 X-MC-Unique: lte1sWzTPa-FCkZeul_B_w-1 Received: by mail-qt1-f197.google.com with SMTP id b14so6845355qtt.1 for ; Fri, 20 Dec 2019 13:03:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5JYh7n3fhb27PXs/bYUs6lzlcg1ya8dgcofIlxRo1Jg=; b=iHn0/+BLKhWj3YmQTqytGOcRFHJezaKzPbXCx7aYxHHPmbhVc2Pngj3nNDv+NdaXUF wwCumfLnKeRW+Ij2MiQXDZNtVY3lFa2FXWUqV5vRDtrF8E9pE5yDB5dnddCiSipf0JA6 v/FNv1l46hM4BaX4xai3bDiOXNST/9LWXswPxx4fHj/79eQPsm55XNCN/jta1PlH3cBC 0HW16xbus/+QeZ+pb0I2fqdvx5oUXZzFRklGpuOnXFhNAelsN2xukeTY10h5Q7dKBrvi qKHdOnt5V9GbdGTZ/LCzo60n9gwEswNM7mbPMjU74jmA3zwiIBrf5u+rldHO80v6B7JD hSSg== X-Gm-Message-State: APjAAAW4GAJCfz4WgEx5AF6/SwUWWKVIvSe2vpQx0PHvOiJ/2ille0sq 8JYYPR3ASd6tEJeHjreDGCog2hXvJk0UmLEOjGt5SnMHaW1sIHafnCD2kzxldrRDSKdgmVSeJwM LgzvEM/sxGxyx X-Received: by 2002:a0c:f8ce:: with SMTP id h14mr14081864qvo.91.1576875816077; Fri, 20 Dec 2019 13:03:36 -0800 (PST) X-Google-Smtp-Source: APXvYqyNSD9sJzoZa53Bscgtcp3kOJukY7A7DFBD+/ym7p7wDcbwYwsje85hFcdBLyzFwr+wgJtPTg== X-Received: by 2002:a0c:f8ce:: with SMTP id h14mr14081841qvo.91.1576875815891; Fri, 20 Dec 2019 13:03:35 -0800 (PST) Received: from xz-x1.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id a9sm3061018qtb.36.2019.12.20.13.03.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2019 13:03:34 -0800 (PST) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterx@redhat.com, Sean Christopherson , "Michael S . Tsirkin" , Vitaly Kuznetsov , Paolo Bonzini , Christophe de Dinechin , "Dr . David Alan Gilbert" , Jason Wang Subject: [PATCH v2 04/17] KVM: Cache as_id in kvm_memory_slot Date: Fri, 20 Dec 2019 16:03:13 -0500 Message-Id: <20191220210326.49949-5-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20191220210326.49949-1-peterx@redhat.com> References: <20191220210326.49949-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Let's cache the address space ID just like the slot ID. Suggested-by: Paolo Bonzini Suggested-by: Sean Christopherson Signed-off-by: Peter Xu --- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 4e34cf97ca90..24854c9e3717 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -348,6 +348,7 @@ struct kvm_memory_slot { unsigned long userspace_addr; u32 flags; short id; + u8 as_id; }; static inline unsigned long kvm_dirty_bitmap_bytes(struct kvm_memory_slot *memslot) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b1047173d78e..cea4b8dd4ac9 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1027,6 +1027,8 @@ int __kvm_set_memory_region(struct kvm *kvm, new = old = *slot; + BUILD_BUG_ON(U8_MAX < KVM_ADDRESS_SPACE_NUM); + new.as_id = as_id; new.id = id; new.base_gfn = base_gfn; new.npages = npages; From patchwork Fri Dec 20 21:03:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11306309 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ED19C14B7 for ; Fri, 20 Dec 2019 21:03:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CB9E5218AC for ; Fri, 20 Dec 2019 21:03:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="R/emlo9R" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727677AbfLTVDr (ORCPT ); Fri, 20 Dec 2019 16:03:47 -0500 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:56284 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727783AbfLTVDm (ORCPT ); Fri, 20 Dec 2019 16:03:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576875821; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vtGsufOEBQRYi5aVidxZ4SgIBK6STwpDg+qKmGEtjBY=; b=R/emlo9R5UQXw8PqKfzhl2ZegQEKn9Y0HiAQyeoXzEFPKroTGAT4nsQXaFfOKhzHMhLUql 8IaK5gVfnq1n7D/1nZTfUYtNFM+Ck05CQa2w9P0fgrjFhtjuzu4CTZ2b5nkAVOJWf0GLbW fJkeIvu/p7tYD6u1Vu6/Nw9QNRmgDrc= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-386-g5s4pMEYPj2-ipEzDn0xrw-1; Fri, 20 Dec 2019 16:03:39 -0500 X-MC-Unique: g5s4pMEYPj2-ipEzDn0xrw-1 Received: by mail-qt1-f197.google.com with SMTP id o18so6783237qtt.19 for ; Fri, 20 Dec 2019 13:03:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vtGsufOEBQRYi5aVidxZ4SgIBK6STwpDg+qKmGEtjBY=; b=UQrXd2J0jS4NQVunaCGWj1+MHFUXn7dTQXV5Be6/Ag0QsbBavsPiAWjfFjQXMxZjXO 9GyzUfA7+sE+J3aOL1Il0sXzjWev/LYZ2mq1cMVhyBThz95rDxYIUeFNzOjYuS17x1iW i0BdRkg6P2xHDvF5kRwTpbfwHN9enYbfXANkTb7XZ9Mhjmvo14OOyFsY0V7CGr1paCkE KpBBmwPuYMNW8vfhlb5OBJDxo3JkbghN69zO450PhytMkjfIQ1NMwH2kne2c4QI63Yl+ n/NxKAlD3W+YD2Eixo5SLWL9EefqapXzvfYNq3yRGIrLSB3aulOt1EfiqdwHalMhtoNt +mdA== X-Gm-Message-State: APjAAAWOnPqundw8E55LO/lyBqGHeH2sgYuHYOVg7ao06kkRWxnYu9L6 LZXxPpfoq4Dfg4NdoHPirZ6hvEm0cmzOtRNCtxrfgL5yoN4w6Nj9UevBuYltJT8LoH/1t0Ua+fz 5Py+SGtuSufJI X-Received: by 2002:ac8:21b5:: with SMTP id 50mr13151560qty.10.1576875818729; Fri, 20 Dec 2019 13:03:38 -0800 (PST) X-Google-Smtp-Source: APXvYqxlRImFhrwNawCgIAgsSOJ1EXj2A5IDQwkpl6IsO1iQYvfDSTKLXH5QiQ2SH4/hegAbB9Mfzg== X-Received: by 2002:ac8:21b5:: with SMTP id 50mr13151548qty.10.1576875818573; Fri, 20 Dec 2019 13:03:38 -0800 (PST) Received: from xz-x1.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id a9sm3061018qtb.36.2019.12.20.13.03.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2019 13:03:36 -0800 (PST) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterx@redhat.com, Sean Christopherson , "Michael S . Tsirkin" , Vitaly Kuznetsov , Paolo Bonzini , Christophe de Dinechin , "Dr . David Alan Gilbert" , Jason Wang Subject: [PATCH v2 05/17] KVM: Add build-time error check on kvm_run size Date: Fri, 20 Dec 2019 16:03:14 -0500 Message-Id: <20191220210326.49949-6-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20191220210326.49949-1-peterx@redhat.com> References: <20191220210326.49949-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org It's already going to reach 2400 Bytes (which is over half of page size on 4K page archs), so maybe it's good to have this build-time check in case it overflows when adding new fields. Signed-off-by: Peter Xu --- virt/kvm/kvm_main.c | 1 + 1 file changed, 1 insertion(+) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index cea4b8dd4ac9..c80a363831ae 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -338,6 +338,7 @@ int kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id) vcpu->pre_pcpu = -1; INIT_LIST_HEAD(&vcpu->blocked_vcpu_list); + BUILD_BUG_ON(sizeof(struct kvm_run) > PAGE_SIZE); page = alloc_page(GFP_KERNEL | __GFP_ZERO); if (!page) { r = -ENOMEM; From patchwork Fri Dec 20 21:03:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11306307 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0B0A14B7 for ; Fri, 20 Dec 2019 21:03:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 94C68218AC for ; Fri, 20 Dec 2019 21:03:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DY6BObZw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727657AbfLTVDp (ORCPT ); Fri, 20 Dec 2019 16:03:45 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:44822 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727791AbfLTVDo (ORCPT ); Fri, 20 Dec 2019 16:03:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576875823; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Pol5PWFpYE9QDZtFO4p/0Ih4s1j3WxPFfstxyW/mAdA=; b=DY6BObZwpbNZmPs8rTOT6EIyAK7uEeoq6ACN1ZqqlWH6K2AAVoIG45HAs9PCtpGbQ27/Oo 4gjWNYPyvTha+RW8YGhCGvN4BoA8R6tjSAMJpDyQVQEJ9SZRw95mlZDdUU3z6woJfBS7R7 XkMCVfJszGe6dr0vEySpboh6j1JwAJc= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-278-6w-vEMEUPQm06gzJzBjFgQ-1; Fri, 20 Dec 2019 16:03:41 -0500 X-MC-Unique: 6w-vEMEUPQm06gzJzBjFgQ-1 Received: by mail-qv1-f72.google.com with SMTP id n11so6715130qvp.15 for ; Fri, 20 Dec 2019 13:03:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Pol5PWFpYE9QDZtFO4p/0Ih4s1j3WxPFfstxyW/mAdA=; b=J7H/jjoHfSpeLexzdNyuLZv8tnmGWMDWfYJzO3/mo23QXXkh0HGBKfu6hNXE5vt2rq u1PitISVptJvGjrCYf0AWz3pRuDxem1XGbDvRGdQxh7L6aIgoRDvPrNL7s1zhjkfom6H Nz5F485FPSUMk0LMPZp7LiS9OFcaaSw0LM8qTLnL+mmApzM3IoURzWB5tlx+yfooCNfc TOaj75NGM/3b0M93QCrI8y7kDIhpBGbvQinZRPoqfWra+YwpEH9APxKxM4l7O+K437u4 5OaGgIlvrhHjnqvBYAxqIR4wpPuoNn/YEGN0aqvIIMTRO39OLHuUxRnK6/D9gxOulsff HcLA== X-Gm-Message-State: APjAAAVzpcuDiOqGPyIDCubvNG+HQxGQxhuvb9fsiISQrdz/YcZokHI3 RJEJ+4in6Up2rgf5BJLpP0YyonfBjssGsqsLrPAeKSnA3kuMQ0LCkC7DbjBULTjfO3uJ83LVkhW pgD5WyINp/Q5z X-Received: by 2002:a05:6214:154b:: with SMTP id t11mr14198261qvw.175.1576875820317; Fri, 20 Dec 2019 13:03:40 -0800 (PST) X-Google-Smtp-Source: APXvYqzGnJV6VBQmJy+GFtTEPIamqoxpbiwGxjBUhIu4gj7U2ESjQ87es1j2xQ7YLZG/dmVH7VrxCg== X-Received: by 2002:a05:6214:154b:: with SMTP id t11mr14198242qvw.175.1576875820067; Fri, 20 Dec 2019 13:03:40 -0800 (PST) Received: from xz-x1.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id a9sm3061018qtb.36.2019.12.20.13.03.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2019 13:03:39 -0800 (PST) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterx@redhat.com, Sean Christopherson , "Michael S . Tsirkin" , Vitaly Kuznetsov , Paolo Bonzini , Christophe de Dinechin , "Dr . David Alan Gilbert" , Jason Wang Subject: [PATCH v2 06/17] KVM: Pass in kvm pointer into mark_page_dirty_in_slot() Date: Fri, 20 Dec 2019 16:03:15 -0500 Message-Id: <20191220210326.49949-7-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20191220210326.49949-1-peterx@redhat.com> References: <20191220210326.49949-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The context will be needed to implement the kvm dirty ring. Signed-off-by: Peter Xu --- virt/kvm/kvm_main.c | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index c80a363831ae..17969cf110dd 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -144,7 +144,9 @@ static void hardware_disable_all(void); static void kvm_io_bus_destroy(struct kvm_io_bus *bus); -static void mark_page_dirty_in_slot(struct kvm_memory_slot *memslot, gfn_t gfn); +static void mark_page_dirty_in_slot(struct kvm *kvm, + struct kvm_memory_slot *memslot, + gfn_t gfn); __visible bool kvm_rebooting; EXPORT_SYMBOL_GPL(kvm_rebooting); @@ -2053,8 +2055,9 @@ int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu, gpa_t gpa, } EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_atomic); -static int __kvm_write_guest_page(struct kvm_memory_slot *memslot, gfn_t gfn, - const void *data, int offset, int len, +static int __kvm_write_guest_page(struct kvm *kvm, + struct kvm_memory_slot *memslot, gfn_t gfn, + const void *data, int offset, int len, bool track_dirty) { int r; @@ -2067,7 +2070,7 @@ static int __kvm_write_guest_page(struct kvm_memory_slot *memslot, gfn_t gfn, if (r) return -EFAULT; if (track_dirty) - mark_page_dirty_in_slot(memslot, gfn); + mark_page_dirty_in_slot(kvm, memslot, gfn); return 0; } @@ -2077,7 +2080,7 @@ int kvm_write_guest_page(struct kvm *kvm, gfn_t gfn, { struct kvm_memory_slot *slot = gfn_to_memslot(kvm, gfn); - return __kvm_write_guest_page(slot, gfn, data, offset, len, + return __kvm_write_guest_page(kvm, slot, gfn, data, offset, len, track_dirty); } EXPORT_SYMBOL_GPL(kvm_write_guest_page); @@ -2087,7 +2090,7 @@ int kvm_vcpu_write_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, { struct kvm_memory_slot *slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn); - return __kvm_write_guest_page(slot, gfn, data, offset, + return __kvm_write_guest_page(vcpu->kvm, slot, gfn, data, offset, len, true); } EXPORT_SYMBOL_GPL(kvm_vcpu_write_guest_page); @@ -2202,7 +2205,7 @@ int kvm_write_guest_offset_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, r = __copy_to_user((void __user *)ghc->hva + offset, data, len); if (r) return -EFAULT; - mark_page_dirty_in_slot(ghc->memslot, gpa >> PAGE_SHIFT); + mark_page_dirty_in_slot(kvm, ghc->memslot, gpa >> PAGE_SHIFT); return 0; } @@ -2269,7 +2272,8 @@ int kvm_clear_guest(struct kvm *kvm, gpa_t gpa, unsigned long len) } EXPORT_SYMBOL_GPL(kvm_clear_guest); -static void mark_page_dirty_in_slot(struct kvm_memory_slot *memslot, +static void mark_page_dirty_in_slot(struct kvm *kvm, + struct kvm_memory_slot *memslot, gfn_t gfn) { if (memslot && memslot->dirty_bitmap) { @@ -2284,7 +2288,7 @@ void mark_page_dirty(struct kvm *kvm, gfn_t gfn) struct kvm_memory_slot *memslot; memslot = gfn_to_memslot(kvm, gfn); - mark_page_dirty_in_slot(memslot, gfn); + mark_page_dirty_in_slot(kvm, memslot, gfn); } EXPORT_SYMBOL_GPL(mark_page_dirty); @@ -2293,7 +2297,7 @@ void kvm_vcpu_mark_page_dirty(struct kvm_vcpu *vcpu, gfn_t gfn) struct kvm_memory_slot *memslot; memslot = kvm_vcpu_gfn_to_memslot(vcpu, gfn); - mark_page_dirty_in_slot(memslot, gfn); + mark_page_dirty_in_slot(vcpu->kvm, memslot, gfn); } EXPORT_SYMBOL_GPL(kvm_vcpu_mark_page_dirty); From patchwork Fri Dec 20 21:03:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11306315 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3773314B7 for ; Fri, 20 Dec 2019 21:04:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0285C21655 for ; Fri, 20 Dec 2019 21:04:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="F5a+kKaX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727640AbfLTVD6 (ORCPT ); Fri, 20 Dec 2019 16:03:58 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:36403 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727584AbfLTVDq (ORCPT ); Fri, 20 Dec 2019 16:03:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576875824; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Xw6vxNv2Dzp8BiJgba6J6ltllpBfM6qD76X6Uw0YGI0=; b=F5a+kKaXsZqQW0FyYKNjXre678N3xrvW8EqCZnYc+2ELJqYT5RzcnwVLy8Qho15o2j8j7f J596EHsB7yAC3fQ+X4+iHtfJ/hD2ZXO1hZl42HC9+i4YVl6ZTkJZAYugRLss38C9nFs36S clGO2Uk4FNUcVvmVFceul0qzpQIK1Vc= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-62-xcRnlVtcOkKPbU1jxuKVyQ-1; Fri, 20 Dec 2019 16:03:43 -0500 X-MC-Unique: xcRnlVtcOkKPbU1jxuKVyQ-1 Received: by mail-qk1-f200.google.com with SMTP id i135so6056694qke.14 for ; Fri, 20 Dec 2019 13:03:43 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Xw6vxNv2Dzp8BiJgba6J6ltllpBfM6qD76X6Uw0YGI0=; b=Sd5zsN+knDvWOq0vTCy7KVQudh8uGwNEH/GQjx1MIe0K08vovxw1z6coKyZCPW8Eum I1O43+a1kVWCpXSmjBoauy6I57koy35T9pMd1t+NmenNw7hjG6yS7HmjdtfGE8n9hAhn E7JkRfSRUmTsBFvfUbSIZX/QaeSJlwCZhvZZXHob+2L1b5ewJNhAKjDdWedZj7IYsGgb IOuseMqMmjDBXqE5axNk/nouLqNbyPq+E2WqZ/DxfwdOln+uY4h3kc/iLN/RUFyfQ/Xd ZLEz9/DPcvqVH/kzoo0tF3vdqea+GbeXv6k0jdlLXwgDLeuudHYa0tMju11DudxYa8CT b4Gw== X-Gm-Message-State: APjAAAWPD/BKAvPSBeJ8RFkJqxbGcxxeROF/G99NsH1GHeEP/XzKoBoy /Nu3C/KoWRuFtDomsvSa4QPWU5NvNStbhxRAMCm6ZAbpUHgAB5QyDwqSyj8GU/i/SZajczO0rjL +qPn51+GTFVoo X-Received: by 2002:ac8:47c1:: with SMTP id d1mr13195928qtr.84.1576875822160; Fri, 20 Dec 2019 13:03:42 -0800 (PST) X-Google-Smtp-Source: APXvYqwMZrDspzkkXPDPXl58tioGBqRjtcG0jGg+abp9ulI7CnPmpZtnITLNx+Ck67xY0/EzS6HkaA== X-Received: by 2002:ac8:47c1:: with SMTP id d1mr13195892qtr.84.1576875821821; Fri, 20 Dec 2019 13:03:41 -0800 (PST) Received: from xz-x1.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id a9sm3061018qtb.36.2019.12.20.13.03.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2019 13:03:41 -0800 (PST) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterx@redhat.com, Sean Christopherson , "Michael S . Tsirkin" , Vitaly Kuznetsov , Paolo Bonzini , Christophe de Dinechin , "Dr . David Alan Gilbert" , Jason Wang Subject: [PATCH v2 07/17] KVM: Move running VCPU from ARM to common code Date: Fri, 20 Dec 2019 16:03:16 -0500 Message-Id: <20191220210326.49949-8-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20191220210326.49949-1-peterx@redhat.com> References: <20191220210326.49949-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Paolo Bonzini For ring-based dirty log tracking, it will be more efficient to account writes during schedule-out or schedule-in to the currently running VCPU. We would like to do it even if the write doesn't use the current VCPU's address space, as is the case for cached writes (see commit 4e335d9e7ddb, "Revert "KVM: Support vCPU-based gfn->hva cache"", 2017-05-02). Therefore, add a mechanism to track the currently-loaded kvm_vcpu struct. There is already something similar in KVM/ARM; one important difference is that kvm_arch_vcpu_{load,put} have two callers in virt/kvm/kvm_main.c: we have to update both the architecture-independent vcpu_{load,put} and the preempt notifiers. Another change made in the process is to allow using kvm_get_running_vcpu() in preemptible code. This is allowed because preempt notifiers ensure that the value does not change even after the VCPU thread is migrated. Signed-off-by: Paolo Bonzini Signed-off-by: Peter Xu --- arch/arm/include/asm/kvm_host.h | 2 -- arch/arm64/include/asm/kvm_host.h | 2 -- include/linux/kvm_host.h | 3 +++ virt/kvm/arm/arch_timer.c | 2 +- virt/kvm/arm/arm.c | 29 ----------------------------- virt/kvm/arm/perf.c | 6 +++--- virt/kvm/arm/vgic/vgic-mmio.c | 15 +++------------ virt/kvm/kvm_main.c | 25 ++++++++++++++++++++++++- 8 files changed, 34 insertions(+), 50 deletions(-) diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h index 8a37c8e89777..40eff9cc3744 100644 --- a/arch/arm/include/asm/kvm_host.h +++ b/arch/arm/include/asm/kvm_host.h @@ -274,8 +274,6 @@ int kvm_arm_copy_reg_indices(struct kvm_vcpu *vcpu, u64 __user *indices); int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end); int kvm_test_age_hva(struct kvm *kvm, unsigned long hva); -struct kvm_vcpu *kvm_arm_get_running_vcpu(void); -struct kvm_vcpu __percpu **kvm_get_running_vcpus(void); void kvm_arm_halt_guest(struct kvm *kvm); void kvm_arm_resume_guest(struct kvm *kvm); diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index f656169db8c3..df8d72f7c20e 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -430,8 +430,6 @@ int kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte); int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end); int kvm_test_age_hva(struct kvm *kvm, unsigned long hva); -struct kvm_vcpu *kvm_arm_get_running_vcpu(void); -struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void); void kvm_arm_halt_guest(struct kvm *kvm); void kvm_arm_resume_guest(struct kvm *kvm); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 24854c9e3717..b4f7bef38e0d 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1323,6 +1323,9 @@ static inline void kvm_vcpu_set_dy_eligible(struct kvm_vcpu *vcpu, bool val) } #endif /* CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT */ +struct kvm_vcpu *kvm_get_running_vcpu(void); +struct kvm_vcpu __percpu **kvm_get_running_vcpus(void); + #ifdef CONFIG_HAVE_KVM_IRQ_BYPASS bool kvm_arch_has_irq_bypass(void); int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *, diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c index e2bb5bd60227..085e7fed850c 100644 --- a/virt/kvm/arm/arch_timer.c +++ b/virt/kvm/arm/arch_timer.c @@ -1022,7 +1022,7 @@ static bool timer_irqs_are_valid(struct kvm_vcpu *vcpu) bool kvm_arch_timer_get_input_level(int vintid) { - struct kvm_vcpu *vcpu = kvm_arm_get_running_vcpu(); + struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); struct arch_timer_context *timer; if (vintid == vcpu_vtimer(vcpu)->irq.irq) diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c index 86c6aa1cb58e..f7dbb94ec525 100644 --- a/virt/kvm/arm/arm.c +++ b/virt/kvm/arm/arm.c @@ -47,9 +47,6 @@ __asm__(".arch_extension virt"); DEFINE_PER_CPU(kvm_host_data_t, kvm_host_data); static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); -/* Per-CPU variable containing the currently running vcpu. */ -static DEFINE_PER_CPU(struct kvm_vcpu *, kvm_arm_running_vcpu); - /* The VMID used in the VTTBR */ static atomic64_t kvm_vmid_gen = ATOMIC64_INIT(1); static u32 kvm_next_vmid; @@ -58,31 +55,8 @@ static DEFINE_SPINLOCK(kvm_vmid_lock); static bool vgic_present; static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled); - -static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu) -{ - __this_cpu_write(kvm_arm_running_vcpu, vcpu); -} - DEFINE_STATIC_KEY_FALSE(userspace_irqchip_in_use); -/** - * kvm_arm_get_running_vcpu - get the vcpu running on the current CPU. - * Must be called from non-preemptible context - */ -struct kvm_vcpu *kvm_arm_get_running_vcpu(void) -{ - return __this_cpu_read(kvm_arm_running_vcpu); -} - -/** - * kvm_arm_get_running_vcpus - get the per-CPU array of currently running vcpus. - */ -struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void) -{ - return &kvm_arm_running_vcpu; -} - int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE; @@ -374,7 +348,6 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) vcpu->cpu = cpu; vcpu->arch.host_cpu_context = &cpu_data->host_ctxt; - kvm_arm_set_running_vcpu(vcpu); kvm_vgic_load(vcpu); kvm_timer_vcpu_load(vcpu); kvm_vcpu_load_sysregs(vcpu); @@ -398,8 +371,6 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) kvm_vcpu_pmu_restore_host(vcpu); vcpu->cpu = -1; - - kvm_arm_set_running_vcpu(NULL); } static void vcpu_power_off(struct kvm_vcpu *vcpu) diff --git a/virt/kvm/arm/perf.c b/virt/kvm/arm/perf.c index 918cdc3839ea..d45b8b9a4415 100644 --- a/virt/kvm/arm/perf.c +++ b/virt/kvm/arm/perf.c @@ -13,14 +13,14 @@ static int kvm_is_in_guest(void) { - return kvm_arm_get_running_vcpu() != NULL; + return kvm_get_running_vcpu() != NULL; } static int kvm_is_user_mode(void) { struct kvm_vcpu *vcpu; - vcpu = kvm_arm_get_running_vcpu(); + vcpu = kvm_get_running_vcpu(); if (vcpu) return !vcpu_mode_priv(vcpu); @@ -32,7 +32,7 @@ static unsigned long kvm_get_guest_ip(void) { struct kvm_vcpu *vcpu; - vcpu = kvm_arm_get_running_vcpu(); + vcpu = kvm_get_running_vcpu(); if (vcpu) return *vcpu_pc(vcpu); diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c index 0d090482720d..d656ebd5f9d4 100644 --- a/virt/kvm/arm/vgic/vgic-mmio.c +++ b/virt/kvm/arm/vgic/vgic-mmio.c @@ -190,15 +190,6 @@ unsigned long vgic_mmio_read_pending(struct kvm_vcpu *vcpu, * value later will give us the same value as we update the per-CPU variable * in the preempt notifier handlers. */ -static struct kvm_vcpu *vgic_get_mmio_requester_vcpu(void) -{ - struct kvm_vcpu *vcpu; - - preempt_disable(); - vcpu = kvm_arm_get_running_vcpu(); - preempt_enable(); - return vcpu; -} /* Must be called with irq->irq_lock held */ static void vgic_hw_irq_spending(struct kvm_vcpu *vcpu, struct vgic_irq *irq, @@ -221,7 +212,7 @@ void vgic_mmio_write_spending(struct kvm_vcpu *vcpu, gpa_t addr, unsigned int len, unsigned long val) { - bool is_uaccess = !vgic_get_mmio_requester_vcpu(); + bool is_uaccess = !kvm_get_running_vcpu(); u32 intid = VGIC_ADDR_TO_INTID(addr, 1); int i; unsigned long flags; @@ -274,7 +265,7 @@ void vgic_mmio_write_cpending(struct kvm_vcpu *vcpu, gpa_t addr, unsigned int len, unsigned long val) { - bool is_uaccess = !vgic_get_mmio_requester_vcpu(); + bool is_uaccess = !kvm_get_running_vcpu(); u32 intid = VGIC_ADDR_TO_INTID(addr, 1); int i; unsigned long flags; @@ -335,7 +326,7 @@ static void vgic_mmio_change_active(struct kvm_vcpu *vcpu, struct vgic_irq *irq, bool active) { unsigned long flags; - struct kvm_vcpu *requester_vcpu = vgic_get_mmio_requester_vcpu(); + struct kvm_vcpu *requester_vcpu = kvm_get_running_vcpu(); raw_spin_lock_irqsave(&irq->irq_lock, flags); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 17969cf110dd..5c606d158854 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -108,6 +108,7 @@ struct kmem_cache *kvm_vcpu_cache; EXPORT_SYMBOL_GPL(kvm_vcpu_cache); static __read_mostly struct preempt_ops kvm_preempt_ops; +static DEFINE_PER_CPU(struct kvm_vcpu *, kvm_running_vcpu); struct dentry *kvm_debugfs_dir; EXPORT_SYMBOL_GPL(kvm_debugfs_dir); @@ -199,6 +200,8 @@ bool kvm_is_reserved_pfn(kvm_pfn_t pfn) void vcpu_load(struct kvm_vcpu *vcpu) { int cpu = get_cpu(); + + __this_cpu_write(kvm_running_vcpu, vcpu); preempt_notifier_register(&vcpu->preempt_notifier); kvm_arch_vcpu_load(vcpu, cpu); put_cpu(); @@ -210,6 +213,7 @@ void vcpu_put(struct kvm_vcpu *vcpu) preempt_disable(); kvm_arch_vcpu_put(vcpu); preempt_notifier_unregister(&vcpu->preempt_notifier); + __this_cpu_write(kvm_running_vcpu, NULL); preempt_enable(); } EXPORT_SYMBOL_GPL(vcpu_put); @@ -4294,8 +4298,8 @@ static void kvm_sched_in(struct preempt_notifier *pn, int cpu) WRITE_ONCE(vcpu->preempted, false); WRITE_ONCE(vcpu->ready, false); + __this_cpu_write(kvm_running_vcpu, vcpu); kvm_arch_sched_in(vcpu, cpu); - kvm_arch_vcpu_load(vcpu, cpu); } @@ -4309,6 +4313,25 @@ static void kvm_sched_out(struct preempt_notifier *pn, WRITE_ONCE(vcpu->ready, true); } kvm_arch_vcpu_put(vcpu); + __this_cpu_write(kvm_running_vcpu, NULL); +} + +/** + * kvm_get_running_vcpu - get the vcpu running on the current CPU. + * Thanks to preempt notifiers, this can also be called from + * preemptible context. + */ +struct kvm_vcpu *kvm_get_running_vcpu(void) +{ + return __this_cpu_read(kvm_running_vcpu); +} + +/** + * kvm_get_running_vcpus - get the per-CPU array of currently running vcpus. + */ +struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void) +{ + return &kvm_running_vcpu; } static void check_processor_compat(void *rtn) From patchwork Fri Dec 20 21:03:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11306311 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A7B5139A for ; Fri, 20 Dec 2019 21:03:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EF61121D7D for ; Fri, 20 Dec 2019 21:03:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="b43BEOUD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727817AbfLTVDv (ORCPT ); Fri, 20 Dec 2019 16:03:51 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:27969 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727506AbfLTVDu (ORCPT ); Fri, 20 Dec 2019 16:03:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576875828; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wEpiTAXa52nbyr0HfKis/BYP1bYtv5k7ov5YPyzos1k=; b=b43BEOUD7l/tJPq5RwjSG3K47/kFvRmJtuORfnEOE4gAgPMokSXCJlsTkDy5M0cBY8ge5X Pcfpi1ScI7QklbWBUQTn0aw6HgFtAIUQoRXMpk54aVbwmPuRayQrd/Jj0iC8xLdzzKFTIk QcNijVIILQHqFMrQV/gG4fCLAXshd7c= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-413-CVQsyOr6O2yMvyRMcTFURA-1; Fri, 20 Dec 2019 16:03:46 -0500 X-MC-Unique: CVQsyOr6O2yMvyRMcTFURA-1 Received: by mail-qt1-f198.google.com with SMTP id l56so6784022qtk.11 for ; Fri, 20 Dec 2019 13:03:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wEpiTAXa52nbyr0HfKis/BYP1bYtv5k7ov5YPyzos1k=; b=h8nQP8MDemynq7ecqQ8I8AiZ5EOvLcQ9NJWE2/QyKLsRd/8tUSQ/JF8mLeXBnwEm/B h2Cpq5TjZSUkoLvDolrSJabgWAFDfGoaCZEf3Q6NBEeoWYQWMfMx1SmwvysWhHTZpzja V+ioCGxsKMR58F/LuUgT/jp7aLQZqpTmlb9+A94bDMzRUNzCZQGkeWxAYQbjbsdzpX+a T1VVCRv5Sisb0C088NsIEoRTKDBBfP5XF7ygTX/8Fqva/M8kELqPyEBCK0zB3Ri1tzSU OeK1aDvh19cAl1hVY1dwzdHKSAIniSZW7FzW8MBtkyAMJWhzG0A2DXSeD1U/bqwtR4pK nyBQ== X-Gm-Message-State: APjAAAVm4cLFPEk3n/PugmL4gripGo0YbBSd6Sh501ZpSMQ6b0vQVD8L 4KtolmMt4Qf3XPvYyh7Nr4zH5rAvr1/YY8lnJz3MC4fnVd7p7ITqBWFMLX78JlWf+G3/lkDYyph 8Dd89l+3nNvU9 X-Received: by 2002:ac8:42de:: with SMTP id g30mr13123143qtm.195.1576875824465; Fri, 20 Dec 2019 13:03:44 -0800 (PST) X-Google-Smtp-Source: APXvYqyL2nsRsQDxAD2WVOUHhufKA0IYeC8t8TU64ZIuSCUIX9gcwfMaxFOPanfm3/6Nqdt+VpYphg== X-Received: by 2002:ac8:42de:: with SMTP id g30mr13123068qtm.195.1576875823458; Fri, 20 Dec 2019 13:03:43 -0800 (PST) Received: from xz-x1.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id a9sm3061018qtb.36.2019.12.20.13.03.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2019 13:03:42 -0800 (PST) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterx@redhat.com, Sean Christopherson , "Michael S . Tsirkin" , Vitaly Kuznetsov , Paolo Bonzini , Christophe de Dinechin , "Dr . David Alan Gilbert" , Jason Wang , Lei Cao Subject: [PATCH v2 08/17] KVM: X86: Implement ring-based dirty memory tracking Date: Fri, 20 Dec 2019 16:03:17 -0500 Message-Id: <20191220210326.49949-9-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20191220210326.49949-1-peterx@redhat.com> References: <20191220210326.49949-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This patch is heavily based on previous work from Lei Cao and Paolo Bonzini . [1] KVM currently uses large bitmaps to track dirty memory. These bitmaps are copied to userspace when userspace queries KVM for its dirty page information. The use of bitmaps is mostly sufficient for live migration, as large parts of memory are be dirtied from one log-dirty pass to another. However, in a checkpointing system, the number of dirty pages is small and in fact it is often bounded---the VM is paused when it has dirtied a pre-defined number of pages. Traversing a large, sparsely populated bitmap to find set bits is time-consuming, as is copying the bitmap to user-space. A similar issue will be there for live migration when the guest memory is huge while the page dirty procedure is trivial. In that case for each dirty sync we need to pull the whole dirty bitmap to userspace and analyse every bit even if it's mostly zeros. The preferred data structure for above scenarios is a dense list of guest frame numbers (GFN). This patch series stores the dirty list in kernel memory that can be memory mapped into userspace to allow speedy harvesting. This patch enables dirty ring for X86 only. However it should be easily extended to other archs as well. [1] https://patchwork.kernel.org/patch/10471409/ Signed-off-by: Lei Cao Signed-off-by: Paolo Bonzini Signed-off-by: Peter Xu --- Documentation/virt/kvm/api.txt | 89 ++++++++++++++ arch/x86/include/asm/kvm_host.h | 3 + arch/x86/include/uapi/asm/kvm.h | 1 + arch/x86/kvm/Makefile | 3 +- arch/x86/kvm/mmu.c | 6 + arch/x86/kvm/vmx/vmx.c | 7 ++ arch/x86/kvm/x86.c | 9 ++ include/linux/kvm_dirty_ring.h | 57 +++++++++ include/linux/kvm_host.h | 28 +++++ include/trace/events/kvm.h | 78 +++++++++++++ include/uapi/linux/kvm.h | 31 +++++ virt/kvm/dirty_ring.c | 201 ++++++++++++++++++++++++++++++++ virt/kvm/kvm_main.c | 172 ++++++++++++++++++++++++++- 13 files changed, 682 insertions(+), 3 deletions(-) create mode 100644 include/linux/kvm_dirty_ring.h create mode 100644 virt/kvm/dirty_ring.c diff --git a/Documentation/virt/kvm/api.txt b/Documentation/virt/kvm/api.txt index 4833904d32a5..c141b285e673 100644 --- a/Documentation/virt/kvm/api.txt +++ b/Documentation/virt/kvm/api.txt @@ -231,6 +231,7 @@ Based on their initialization different VMs may have different capabilities. It is thus encouraged to use the vm ioctl to query for capabilities (available with KVM_CAP_CHECK_EXTENSION_VM on the vm fd) + 4.5 KVM_GET_VCPU_MMAP_SIZE Capability: basic @@ -243,6 +244,18 @@ The KVM_RUN ioctl (cf.) communicates with userspace via a shared memory region. This ioctl returns the size of that region. See the KVM_RUN documentation for details. +Besides the size of the KVM_RUN communication region, other areas of +the VCPU file descriptor can be mmap-ed, including: + +- if KVM_CAP_COALESCED_MMIO is available, a page at + KVM_COALESCED_MMIO_PAGE_OFFSET * PAGE_SIZE; for historical reasons, + this page is included in the result of KVM_GET_VCPU_MMAP_SIZE. + KVM_CAP_COALESCED_MMIO is not documented yet. + +- if KVM_CAP_DIRTY_LOG_RING is available, a number of pages at + KVM_DIRTY_LOG_PAGE_OFFSET * PAGE_SIZE. For more information on + KVM_CAP_DIRTY_LOG_RING, see section 8.3. + 4.6 KVM_SET_MEMORY_REGION @@ -5302,6 +5315,7 @@ CPU when the exception is taken. If this virtual SError is taken to EL1 using AArch64, this value will be reported in the ISS field of ESR_ELx. See KVM_CAP_VCPU_EVENTS for more details. + 8.20 KVM_CAP_HYPERV_SEND_IPI Architectures: x86 @@ -5309,6 +5323,7 @@ Architectures: x86 This capability indicates that KVM supports paravirtualized Hyper-V IPI send hypercalls: HvCallSendSyntheticClusterIpi, HvCallSendSyntheticClusterIpiEx. + 8.21 KVM_CAP_HYPERV_DIRECT_TLBFLUSH Architecture: x86 @@ -5322,3 +5337,77 @@ handling by KVM (as some KVM hypercall may be mistakenly treated as TLB flush hypercalls by Hyper-V) so userspace should disable KVM identification in CPUID and only exposes Hyper-V identification. In this case, guest thinks it's running on Hyper-V and only use Hyper-V hypercalls. + +8.22 KVM_CAP_DIRTY_LOG_RING + +Architectures: x86 +Parameters: args[0] - size of the dirty log ring + +KVM is capable of tracking dirty memory using ring buffers that are +mmaped into userspace; there is one dirty ring per vcpu. + +One dirty ring is defined as below internally: + +struct kvm_dirty_ring { + u32 dirty_index; + u32 reset_index; + u32 size; + u32 soft_limit; + struct kvm_dirty_gfn *dirty_gfns; + struct kvm_dirty_ring_indices *indices; + int index; +}; + +Dirty GFNs (Guest Frame Numbers) are stored in the dirty_gfns array. +For each of the dirty entry it's defined as: + +struct kvm_dirty_gfn { + __u32 pad; + __u32 slot; /* as_id | slot_id */ + __u64 offset; +}; + +Most of the ring structure is used by KVM internally, while only the +indices are exposed to userspace: + +struct kvm_dirty_ring_indices { + __u32 avail_index; /* set by kernel */ + __u32 fetch_index; /* set by userspace */ +}; + +The two indices in the ring buffer are free running counters. + +Userspace calls KVM_ENABLE_CAP ioctl right after KVM_CREATE_VM ioctl +to enable this capability for the new guest and set the size of the +rings. It is only allowed before creating any vCPU, and the size of +the ring must be a power of two. The larger the ring buffer, the less +likely the ring is full and the VM is forced to exit to userspace. The +optimal size depends on the workload, but it is recommended that it be +at least 64 KiB (4096 entries). + +Just like for dirty page bitmaps, the buffer tracks writes to +all user memory regions for which the KVM_MEM_LOG_DIRTY_PAGES flag was +set in KVM_SET_USER_MEMORY_REGION. Once a memory region is registered +with the flag set, userspace can start harvesting dirty pages from the +ring buffer. + +To harvest the dirty pages, userspace accesses the mmaped ring buffer +to read the dirty GFNs up to avail_index, and sets the fetch_index +accordingly. This can be done when the guest is running or paused, +and dirty pages need not be collected all at once. After processing +one or more entries in the ring buffer, userspace calls the VM ioctl +KVM_RESET_DIRTY_RINGS to notify the kernel that it has updated +fetch_index and to mark those pages clean. Therefore, the ioctl +must be called *before* reading the content of the dirty pages. + +However, there is a major difference comparing to the +KVM_GET_DIRTY_LOG interface in that when reading the dirty ring from +userspace it's still possible that the kernel has not yet flushed the +hardware dirty buffers into the kernel buffer (which was previously +done by the KVM_GET_DIRTY_LOG ioctl). To achieve that, one needs to +kick the vcpu out for a hardware buffer flush (vmexit) to make sure +all the existing dirty gfns are flushed to the dirty rings. + +If one of the ring buffers is full, the guest will exit to userspace +with the exit reason set to KVM_EXIT_DIRTY_LOG_FULL, and the KVM_RUN +ioctl will return to userspace with zero. diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 4fc61483919a..7e5e2d3f0509 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1159,6 +1159,7 @@ struct kvm_x86_ops { struct kvm_memory_slot *slot, gfn_t offset, unsigned long mask); int (*write_log_dirty)(struct kvm_vcpu *vcpu); + int (*cpu_dirty_log_size)(void); /* pmu operations of sub-arch */ const struct kvm_pmu_ops *pmu_ops; @@ -1641,4 +1642,6 @@ static inline int kvm_cpu_get_apicid(int mps_cpu) #define GET_SMSTATE(type, buf, offset) \ (*(type *)((buf) + (offset) - 0x7e00)) +int kvm_cpu_dirty_log_size(void); + #endif /* _ASM_X86_KVM_HOST_H */ diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 503d3f42da16..b59bf356c478 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -12,6 +12,7 @@ #define KVM_PIO_PAGE_OFFSET 1 #define KVM_COALESCED_MMIO_PAGE_OFFSET 2 +#define KVM_DIRTY_LOG_PAGE_OFFSET 64 #define DE_VECTOR 0 #define DB_VECTOR 1 diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile index 31ecf7a76d5a..a66ddb552208 100644 --- a/arch/x86/kvm/Makefile +++ b/arch/x86/kvm/Makefile @@ -5,7 +5,8 @@ ccflags-y += -Iarch/x86/kvm KVM := ../../../virt/kvm kvm-y += $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o \ - $(KVM)/eventfd.o $(KVM)/irqchip.o $(KVM)/vfio.o + $(KVM)/eventfd.o $(KVM)/irqchip.o $(KVM)/vfio.o \ + $(KVM)/dirty_ring.o kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o kvm-y += x86.o mmu.o emulate.o i8259.o irq.o lapic.o \ diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index 2ce9da58611e..5f7d73730f73 100644 --- a/arch/x86/kvm/mmu.c +++ b/arch/x86/kvm/mmu.c @@ -1818,7 +1818,13 @@ int kvm_arch_write_log_dirty(struct kvm_vcpu *vcpu) { if (kvm_x86_ops->write_log_dirty) return kvm_x86_ops->write_log_dirty(vcpu); + return 0; +} +int kvm_cpu_dirty_log_size(void) +{ + if (kvm_x86_ops->cpu_dirty_log_size) + return kvm_x86_ops->cpu_dirty_log_size(); return 0; } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 1ff5a428f489..c3565319b481 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7686,6 +7686,7 @@ static __init int hardware_setup(void) kvm_x86_ops->slot_disable_log_dirty = NULL; kvm_x86_ops->flush_log_dirty = NULL; kvm_x86_ops->enable_log_dirty_pt_masked = NULL; + kvm_x86_ops->cpu_dirty_log_size = NULL; } if (!cpu_has_vmx_preemption_timer()) @@ -7750,6 +7751,11 @@ static __exit void hardware_unsetup(void) free_kvm_area(); } +static int vmx_cpu_dirty_log_size(void) +{ + return enable_pml ? PML_ENTITY_NUM : 0; +} + static struct kvm_x86_ops vmx_x86_ops __ro_after_init = { .cpu_has_kvm_support = cpu_has_kvm_support, .disabled_by_bios = vmx_disabled_by_bios, @@ -7873,6 +7879,7 @@ static struct kvm_x86_ops vmx_x86_ops __ro_after_init = { .flush_log_dirty = vmx_flush_log_dirty, .enable_log_dirty_pt_masked = vmx_enable_log_dirty_pt_masked, .write_log_dirty = vmx_write_pml_buffer, + .cpu_dirty_log_size = vmx_cpu_dirty_log_size, .pre_block = vmx_pre_block, .post_block = vmx_post_block, diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 5d530521f11d..f93262025a61 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -7965,6 +7965,15 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) bool req_immediate_exit = false; + /* Forbid vmenter if vcpu dirty ring is soft-full */ + if (unlikely(vcpu->kvm->dirty_ring_size && + kvm_dirty_ring_soft_full(&vcpu->dirty_ring))) { + vcpu->run->exit_reason = KVM_EXIT_DIRTY_RING_FULL; + trace_kvm_dirty_ring_exit(vcpu); + r = 0; + goto out; + } + if (kvm_request_pending(vcpu)) { if (kvm_check_request(KVM_REQ_GET_VMCS12_PAGES, vcpu)) { if (unlikely(!kvm_x86_ops->get_vmcs12_pages(vcpu))) { diff --git a/include/linux/kvm_dirty_ring.h b/include/linux/kvm_dirty_ring.h new file mode 100644 index 000000000000..06db2312b383 --- /dev/null +++ b/include/linux/kvm_dirty_ring.h @@ -0,0 +1,57 @@ +#ifndef KVM_DIRTY_RING_H +#define KVM_DIRTY_RING_H + +/** + * kvm_dirty_ring: KVM internal dirty ring structure + * + * @dirty_index: free running counter that points to the next slot in + * dirty_ring->dirty_gfns, where a new dirty page should go + * @reset_index: free running counter that points to the next dirty page + * in dirty_ring->dirty_gfns for which dirty trap needs to + * be reenabled + * @size: size of the compact list, dirty_ring->dirty_gfns + * @soft_limit: when the number of dirty pages in the list reaches this + * limit, vcpu that owns this ring should exit to userspace + * to allow userspace to harvest all the dirty pages + * @dirty_gfns: the array to keep the dirty gfns + * @indices: the pointer to the @kvm_dirty_ring_indices structure + * of this specific ring + * @index: index of this dirty ring + */ +struct kvm_dirty_ring { + u32 dirty_index; + u32 reset_index; + u32 size; + u32 soft_limit; + struct kvm_dirty_gfn *dirty_gfns; + struct kvm_dirty_ring_indices *indices; + int index; +}; + +u32 kvm_dirty_ring_get_rsvd_entries(void); +int kvm_dirty_ring_alloc(struct kvm_dirty_ring *ring, + struct kvm_dirty_ring_indices *indices, + int index, u32 size); +struct kvm_dirty_ring *kvm_dirty_ring_get(struct kvm *kvm); +void kvm_dirty_ring_put(struct kvm *kvm, + struct kvm_dirty_ring *ring); + +/* + * called with kvm->slots_lock held, returns the number of + * processed pages. + */ +int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring); + +/* + * returns =0: successfully pushed + * <0: unable to push, need to wait + */ +int kvm_dirty_ring_push(struct kvm_dirty_ring *ring, u32 slot, u64 offset); + +/* for use in vm_operations_struct */ +struct page *kvm_dirty_ring_get_page(struct kvm_dirty_ring *ring, u32 offset); + +void kvm_dirty_ring_free(struct kvm_dirty_ring *ring); +bool kvm_dirty_ring_soft_full(struct kvm_dirty_ring *ring); + +#endif diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index b4f7bef38e0d..dff214ab72eb 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -34,6 +34,7 @@ #include #include +#include #ifndef KVM_MAX_VCPU_ID #define KVM_MAX_VCPU_ID KVM_MAX_VCPUS @@ -321,6 +322,7 @@ struct kvm_vcpu { bool ready; struct kvm_vcpu_arch arch; struct dentry *debugfs_dentry; + struct kvm_dirty_ring dirty_ring; }; static inline int kvm_vcpu_exiting_guest_mode(struct kvm_vcpu *vcpu) @@ -502,6 +504,9 @@ struct kvm { struct srcu_struct srcu; struct srcu_struct irq_srcu; pid_t userspace_pid; + u32 dirty_ring_size; + struct spinlock dirty_ring_lock; + wait_queue_head_t dirty_ring_waitqueue; }; #define kvm_err(fmt, ...) \ @@ -813,6 +818,8 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm, gfn_t gfn_offset, unsigned long mask); +void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask); + int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log); int kvm_vm_ioctl_clear_dirty_log(struct kvm *kvm, @@ -1392,4 +1399,25 @@ int kvm_vm_create_worker_thread(struct kvm *kvm, kvm_vm_thread_fn_t thread_fn, uintptr_t data, const char *name, struct task_struct **thread_ptr); +/* + * This defines how many reserved entries we want to keep before we + * kick the vcpu to the userspace to avoid dirty ring full. This + * value can be tuned to higher if e.g. PML is enabled on the host. + */ +#define KVM_DIRTY_RING_RSVD_ENTRIES 64 + +/* Max number of entries allowed for each kvm dirty ring */ +#define KVM_DIRTY_RING_MAX_ENTRIES 65536 + +/* + * Arch needs to define these macro after implementing the dirty ring + * feature. KVM_DIRTY_LOG_PAGE_OFFSET should be defined as the + * starting page offset of the dirty ring structures, while + * KVM_DIRTY_RING_VERSION should be defined as >=1. By default, this + * feature is off on all archs. + */ +#ifndef KVM_DIRTY_LOG_PAGE_OFFSET +#define KVM_DIRTY_LOG_PAGE_OFFSET 0 +#endif + #endif diff --git a/include/trace/events/kvm.h b/include/trace/events/kvm.h index 2c735a3e6613..3d850997940c 100644 --- a/include/trace/events/kvm.h +++ b/include/trace/events/kvm.h @@ -399,6 +399,84 @@ TRACE_EVENT(kvm_halt_poll_ns, #define trace_kvm_halt_poll_ns_shrink(vcpu_id, new, old) \ trace_kvm_halt_poll_ns(false, vcpu_id, new, old) +TRACE_EVENT(kvm_dirty_ring_push, + TP_PROTO(struct kvm_dirty_ring *ring, u32 slot, u64 offset), + TP_ARGS(ring, slot, offset), + + TP_STRUCT__entry( + __field(int, index) + __field(u32, dirty_index) + __field(u32, reset_index) + __field(u32, slot) + __field(u64, offset) + ), + + TP_fast_assign( + __entry->index = ring->index; + __entry->dirty_index = ring->dirty_index; + __entry->reset_index = ring->reset_index; + __entry->slot = slot; + __entry->offset = offset; + ), + + TP_printk("ring %d: dirty 0x%x reset 0x%x " + "slot %u offset 0x%llx (used %u)", + __entry->index, __entry->dirty_index, + __entry->reset_index, __entry->slot, __entry->offset, + __entry->dirty_index - __entry->reset_index) +); + +TRACE_EVENT(kvm_dirty_ring_reset, + TP_PROTO(struct kvm_dirty_ring *ring), + TP_ARGS(ring), + + TP_STRUCT__entry( + __field(int, index) + __field(u32, dirty_index) + __field(u32, reset_index) + ), + + TP_fast_assign( + __entry->index = ring->index; + __entry->dirty_index = ring->dirty_index; + __entry->reset_index = ring->reset_index; + ), + + TP_printk("ring %d: dirty 0x%x reset 0x%x (used %u)", + __entry->index, __entry->dirty_index, __entry->reset_index, + __entry->dirty_index - __entry->reset_index) +); + +TRACE_EVENT(kvm_dirty_ring_waitqueue, + TP_PROTO(bool enter), + TP_ARGS(enter), + + TP_STRUCT__entry( + __field(bool, enter) + ), + + TP_fast_assign( + __entry->enter = enter; + ), + + TP_printk("%s", __entry->enter ? "wait" : "awake") +); + +TRACE_EVENT(kvm_dirty_ring_exit, + TP_PROTO(struct kvm_vcpu *vcpu), + TP_ARGS(vcpu), + + TP_STRUCT__entry( + __field(int, vcpu_id) + ), + + TP_fast_assign( + __entry->vcpu_id = vcpu->vcpu_id; + ), + + TP_printk("vcpu %d", __entry->vcpu_id) +); + #endif /* _TRACE_KVM_MAIN_H */ /* This part must be outside protection */ diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 52641d8ca9e8..5ea98e35a129 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -235,6 +235,7 @@ struct kvm_hyperv_exit { #define KVM_EXIT_S390_STSI 25 #define KVM_EXIT_IOAPIC_EOI 26 #define KVM_EXIT_HYPERV 27 +#define KVM_EXIT_DIRTY_RING_FULL 28 /* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */ @@ -246,6 +247,11 @@ struct kvm_hyperv_exit { /* Encounter unexpected vm-exit reason */ #define KVM_INTERNAL_ERROR_UNEXPECTED_EXIT_REASON 4 +struct kvm_dirty_ring_indices { + __u32 avail_index; /* set by kernel */ + __u32 fetch_index; /* set by userspace */ +}; + /* for KVM_RUN, returned by mmap(vcpu_fd, offset=0) */ struct kvm_run { /* in */ @@ -415,6 +421,8 @@ struct kvm_run { struct kvm_sync_regs regs; char padding[SYNC_REGS_SIZE_BYTES]; } s; + + struct kvm_dirty_ring_indices vcpu_ring_indices; }; /* for KVM_REGISTER_COALESCED_MMIO / KVM_UNREGISTER_COALESCED_MMIO */ @@ -1000,6 +1008,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_PMU_EVENT_FILTER 173 #define KVM_CAP_ARM_IRQ_LINE_LAYOUT_2 174 #define KVM_CAP_HYPERV_DIRECT_TLBFLUSH 175 +#define KVM_CAP_DIRTY_LOG_RING 176 #ifdef KVM_CAP_IRQ_ROUTING @@ -1461,6 +1470,9 @@ struct kvm_enc_region { /* Available with KVM_CAP_ARM_SVE */ #define KVM_ARM_VCPU_FINALIZE _IOW(KVMIO, 0xc2, int) +/* Available with KVM_CAP_DIRTY_LOG_RING */ +#define KVM_RESET_DIRTY_RINGS _IO(KVMIO, 0xc3) + /* Secure Encrypted Virtualization command */ enum sev_cmd_id { /* Guest initialization commands */ @@ -1611,4 +1623,23 @@ struct kvm_hyperv_eventfd { #define KVM_HYPERV_CONN_ID_MASK 0x00ffffff #define KVM_HYPERV_EVENTFD_DEASSIGN (1 << 0) +/* + * The following are the requirements for supporting dirty log ring + * (by enabling KVM_DIRTY_LOG_PAGE_OFFSET). + * + * 1. Memory accesses by KVM should call kvm_vcpu_write_* instead + * of kvm_write_* so that the global dirty ring is not filled up + * too quickly. + * 2. kvm_arch_mmu_enable_log_dirty_pt_masked should be defined for + * enabling dirty logging. + * 3. There should not be a separate step to synchronize hardware + * dirty bitmap with KVM's. + */ + +struct kvm_dirty_gfn { + __u32 pad; + __u32 slot; + __u64 offset; +}; + #endif /* __LINUX_KVM_H */ diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c new file mode 100644 index 000000000000..c614822493ff --- /dev/null +++ b/virt/kvm/dirty_ring.c @@ -0,0 +1,201 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * KVM dirty ring implementation + * + * Copyright 2019 Red Hat, Inc. + */ +#include +#include +#include +#include +#include + +int __weak kvm_cpu_dirty_log_size(void) +{ + return 0; +} + +u32 kvm_dirty_ring_get_rsvd_entries(void) +{ + return KVM_DIRTY_RING_RSVD_ENTRIES + kvm_cpu_dirty_log_size(); +} + +static u32 kvm_dirty_ring_used(struct kvm_dirty_ring *ring) +{ + return READ_ONCE(ring->dirty_index) - READ_ONCE(ring->reset_index); +} + +bool kvm_dirty_ring_soft_full(struct kvm_dirty_ring *ring) +{ + return kvm_dirty_ring_used(ring) >= ring->soft_limit; +} + +bool kvm_dirty_ring_full(struct kvm_dirty_ring *ring) +{ + return kvm_dirty_ring_used(ring) >= ring->size; +} + +struct kvm_dirty_ring *kvm_dirty_ring_get(struct kvm *kvm) +{ + struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); + + /* + * TODO: Currently use vcpu0 as default ring. Note that this + * should not happen only if called by kvmgt_rw_gpa for x86. + * After the kvmgt code refactoring we should remove this, + * together with the kvm->dirty_ring_lock. + */ + if (!vcpu) { + pr_warn_once("Detected page dirty without vcpu context. " + "Probably because kvm-gt is used. " + "May expect unbalanced loads on vcpu0."); + vcpu = kvm->vcpus[0]; + } + + WARN_ON_ONCE(vcpu->kvm != kvm); + + if (vcpu == kvm->vcpus[0]) + spin_lock(&kvm->dirty_ring_lock); + + return &vcpu->dirty_ring; +} + +void kvm_dirty_ring_put(struct kvm *kvm, + struct kvm_dirty_ring *ring) +{ + struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); + + if (!vcpu) + vcpu = kvm->vcpus[0]; + + WARN_ON_ONCE(vcpu->kvm != kvm); + WARN_ON_ONCE(&vcpu->dirty_ring != ring); + + if (vcpu == kvm->vcpus[0]) + spin_unlock(&kvm->dirty_ring_lock); +} + +int kvm_dirty_ring_alloc(struct kvm_dirty_ring *ring, + struct kvm_dirty_ring_indices *indices, + int index, u32 size) +{ + ring->dirty_gfns = vmalloc(size); + if (!ring->dirty_gfns) + return -ENOMEM; + memset(ring->dirty_gfns, 0, size); + + ring->size = size / sizeof(struct kvm_dirty_gfn); + ring->soft_limit = ring->size - kvm_dirty_ring_get_rsvd_entries(); + ring->dirty_index = 0; + ring->reset_index = 0; + ring->index = index; + ring->indices = indices; + + return 0; +} + +int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring) +{ + u32 cur_slot, next_slot; + u64 cur_offset, next_offset; + unsigned long mask; + u32 fetch; + int count = 0; + struct kvm_dirty_gfn *entry; + struct kvm_dirty_ring_indices *indices = ring->indices; + bool first_round = true; + + fetch = READ_ONCE(indices->fetch_index); + + /* + * Note that fetch_index is written by the userspace, which + * should not be trusted. If this happens, then it's probably + * that the userspace has written a wrong fetch_index. + */ + if (fetch - ring->reset_index > ring->size) + return -EINVAL; + + if (fetch == ring->reset_index) + return 0; + + /* This is only needed to make compilers happy */ + cur_slot = cur_offset = mask = 0; + while (ring->reset_index != fetch) { + entry = &ring->dirty_gfns[ring->reset_index & (ring->size - 1)]; + next_slot = READ_ONCE(entry->slot); + next_offset = READ_ONCE(entry->offset); + ring->reset_index++; + count++; + /* + * Try to coalesce the reset operations when the guest is + * scanning pages in the same slot. + */ + if (!first_round && next_slot == cur_slot) { + s64 delta = next_offset - cur_offset; + + if (delta >= 0 && delta < BITS_PER_LONG) { + mask |= 1ull << delta; + continue; + } + + /* Backwards visit, careful about overflows! */ + if (delta > -BITS_PER_LONG && delta < 0 && + (mask << -delta >> -delta) == mask) { + cur_offset = next_offset; + mask = (mask << -delta) | 1; + continue; + } + } + kvm_reset_dirty_gfn(kvm, cur_slot, cur_offset, mask); + cur_slot = next_slot; + cur_offset = next_offset; + mask = 1; + first_round = false; + } + kvm_reset_dirty_gfn(kvm, cur_slot, cur_offset, mask); + + trace_kvm_dirty_ring_reset(ring); + + return count; +} + +int kvm_dirty_ring_push(struct kvm_dirty_ring *ring, u32 slot, u64 offset) +{ + struct kvm_dirty_gfn *entry; + struct kvm_dirty_ring_indices *indices = ring->indices; + + /* + * Note: here we will start waiting even soft full, because we + * can't risk making it completely full, since vcpu0 could use + * it right after us and if vcpu0 context gets full it could + * deadlock if wait with mmu_lock held. + */ + if (kvm_get_running_vcpu() == NULL && + kvm_dirty_ring_soft_full(ring)) + return -EBUSY; + + /* It will never gets completely full when with a vcpu context */ + WARN_ON_ONCE(kvm_dirty_ring_full(ring)); + + entry = &ring->dirty_gfns[ring->dirty_index & (ring->size - 1)]; + entry->slot = slot; + entry->offset = offset; + smp_wmb(); + ring->dirty_index++; + WRITE_ONCE(indices->avail_index, ring->dirty_index); + + trace_kvm_dirty_ring_push(ring, slot, offset); + + return 0; +} + +struct page *kvm_dirty_ring_get_page(struct kvm_dirty_ring *ring, u32 offset) +{ + return vmalloc_to_page((void *)ring->dirty_gfns + offset * PAGE_SIZE); +} + +void kvm_dirty_ring_free(struct kvm_dirty_ring *ring) +{ + vfree(ring->dirty_gfns); + ring->dirty_gfns = NULL; +} diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 5c606d158854..4050631d05f3 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -64,6 +64,8 @@ #define CREATE_TRACE_POINTS #include +#include + /* Worst case buffer size needed for holding an integer. */ #define ITOA_MAX_LEN 12 @@ -148,6 +150,9 @@ static void kvm_io_bus_destroy(struct kvm_io_bus *bus); static void mark_page_dirty_in_slot(struct kvm *kvm, struct kvm_memory_slot *memslot, gfn_t gfn); +static void mark_page_dirty_in_ring(struct kvm *kvm, + struct kvm_memory_slot *slot, + gfn_t gfn); __visible bool kvm_rebooting; EXPORT_SYMBOL_GPL(kvm_rebooting); @@ -357,11 +362,22 @@ int kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id) vcpu->preempted = false; vcpu->ready = false; + if (kvm->dirty_ring_size) { + r = kvm_dirty_ring_alloc(&vcpu->dirty_ring, + &vcpu->run->vcpu_ring_indices, + id, kvm->dirty_ring_size); + if (r) + goto fail_free_run; + } + r = kvm_arch_vcpu_init(vcpu); if (r < 0) - goto fail_free_run; + goto fail_free_ring; return 0; +fail_free_ring: + if (kvm->dirty_ring_size) + kvm_dirty_ring_free(&vcpu->dirty_ring); fail_free_run: free_page((unsigned long)vcpu->run); fail: @@ -379,6 +395,8 @@ void kvm_vcpu_uninit(struct kvm_vcpu *vcpu) put_pid(rcu_dereference_protected(vcpu->pid, 1)); kvm_arch_vcpu_uninit(vcpu); free_page((unsigned long)vcpu->run); + if (vcpu->kvm->dirty_ring_size) + kvm_dirty_ring_free(&vcpu->dirty_ring); } EXPORT_SYMBOL_GPL(kvm_vcpu_uninit); @@ -693,6 +711,7 @@ static struct kvm *kvm_create_vm(unsigned long type) return ERR_PTR(-ENOMEM); spin_lock_init(&kvm->mmu_lock); + spin_lock_init(&kvm->dirty_ring_lock); mmgrab(current->mm); kvm->mm = current->mm; kvm_eventfd_init(kvm); @@ -700,6 +719,7 @@ static struct kvm *kvm_create_vm(unsigned long type) mutex_init(&kvm->irq_lock); mutex_init(&kvm->slots_lock); INIT_LIST_HEAD(&kvm->devices); + init_waitqueue_head(&kvm->dirty_ring_waitqueue); BUILD_BUG_ON(KVM_MEM_SLOTS_NUM > SHRT_MAX); @@ -2283,7 +2303,10 @@ static void mark_page_dirty_in_slot(struct kvm *kvm, if (memslot && memslot->dirty_bitmap) { unsigned long rel_gfn = gfn - memslot->base_gfn; - set_bit_le(rel_gfn, memslot->dirty_bitmap); + if (kvm->dirty_ring_size) + mark_page_dirty_in_ring(kvm, memslot, gfn); + else + set_bit_le(rel_gfn, memslot->dirty_bitmap); } } @@ -2630,6 +2653,16 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me, bool yield_to_kernel_mode) } EXPORT_SYMBOL_GPL(kvm_vcpu_on_spin); +static bool kvm_fault_in_dirty_ring(struct kvm *kvm, struct vm_fault *vmf) +{ + if (!KVM_DIRTY_LOG_PAGE_OFFSET) + return false; + + return (vmf->pgoff >= KVM_DIRTY_LOG_PAGE_OFFSET) && + (vmf->pgoff < KVM_DIRTY_LOG_PAGE_OFFSET + + kvm->dirty_ring_size / PAGE_SIZE); +} + static vm_fault_t kvm_vcpu_fault(struct vm_fault *vmf) { struct kvm_vcpu *vcpu = vmf->vma->vm_file->private_data; @@ -2645,6 +2678,10 @@ static vm_fault_t kvm_vcpu_fault(struct vm_fault *vmf) else if (vmf->pgoff == KVM_COALESCED_MMIO_PAGE_OFFSET) page = virt_to_page(vcpu->kvm->coalesced_mmio_ring); #endif + else if (kvm_fault_in_dirty_ring(vcpu->kvm, vmf)) + page = kvm_dirty_ring_get_page( + &vcpu->dirty_ring, + vmf->pgoff - KVM_DIRTY_LOG_PAGE_OFFSET); else return kvm_arch_vcpu_fault(vcpu, vmf); get_page(page); @@ -3239,12 +3276,138 @@ static long kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg) #endif case KVM_CAP_NR_MEMSLOTS: return KVM_USER_MEM_SLOTS; + case KVM_CAP_DIRTY_LOG_RING: +#ifdef CONFIG_X86 + return KVM_DIRTY_RING_MAX_ENTRIES; +#else + return 0; +#endif default: break; } return kvm_vm_ioctl_check_extension(kvm, arg); } +static void mark_page_dirty_in_ring(struct kvm *kvm, + struct kvm_memory_slot *slot, + gfn_t gfn) +{ + struct kvm_dirty_ring *ring; + u64 offset; + int ret; + + if (!kvm->dirty_ring_size) + return; + + offset = gfn - slot->base_gfn; + + ring = kvm_dirty_ring_get(kvm); + +retry: + ret = kvm_dirty_ring_push(ring, (slot->as_id << 16) | slot->id, + offset); + if (ret < 0) { + /* We must be without a vcpu context. */ + WARN_ON_ONCE(kvm_get_running_vcpu()); + + trace_kvm_dirty_ring_waitqueue(1); + /* + * Ring is full, put us onto per-vm waitqueue and wait + * for another KVM_RESET_DIRTY_RINGS to retry + */ + wait_event_killable(kvm->dirty_ring_waitqueue, + !kvm_dirty_ring_soft_full(ring)); + + trace_kvm_dirty_ring_waitqueue(0); + + /* If we're killed, no worry on lossing dirty bits */ + if (fatal_signal_pending(current)) + return; + + goto retry; + } + + kvm_dirty_ring_put(kvm, ring); +} + +void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask) +{ + struct kvm_memory_slot *memslot; + int as_id, id; + + as_id = slot >> 16; + id = (u16)slot; + if (as_id >= KVM_ADDRESS_SPACE_NUM || id >= KVM_USER_MEM_SLOTS) + return; + + memslot = id_to_memslot(__kvm_memslots(kvm, as_id), id); + if (offset >= memslot->npages) + return; + + spin_lock(&kvm->mmu_lock); + kvm_arch_mmu_enable_log_dirty_pt_masked(kvm, memslot, offset, mask); + spin_unlock(&kvm->mmu_lock); +} + +static int kvm_vm_ioctl_enable_dirty_log_ring(struct kvm *kvm, u32 size) +{ + int r; + + /* the size should be power of 2 */ + if (!size || (size & (size - 1))) + return -EINVAL; + + /* Should be bigger to keep the reserved entries, or a page */ + if (size < kvm_dirty_ring_get_rsvd_entries() * + sizeof(struct kvm_dirty_gfn) || size < PAGE_SIZE) + return -EINVAL; + + if (size > KVM_DIRTY_RING_MAX_ENTRIES * + sizeof(struct kvm_dirty_gfn)) + return -E2BIG; + + /* We only allow it to set once */ + if (kvm->dirty_ring_size) + return -EINVAL; + + mutex_lock(&kvm->lock); + + if (kvm->created_vcpus) { + /* We don't allow to change this value after vcpu created */ + r = -EINVAL; + } else { + kvm->dirty_ring_size = size; + r = 0; + } + + mutex_unlock(&kvm->lock); + return r; +} + +static int kvm_vm_ioctl_reset_dirty_pages(struct kvm *kvm) +{ + int i; + struct kvm_vcpu *vcpu; + int cleared = 0; + + if (!kvm->dirty_ring_size) + return -EINVAL; + + mutex_lock(&kvm->slots_lock); + + kvm_for_each_vcpu(i, vcpu, kvm) + cleared += kvm_dirty_ring_reset(vcpu->kvm, &vcpu->dirty_ring); + + mutex_unlock(&kvm->slots_lock); + + if (cleared) + kvm_flush_remote_tlbs(kvm); + + wake_up_all(&kvm->dirty_ring_waitqueue); + + return cleared; +} + int __attribute__((weak)) kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { @@ -3262,6 +3425,8 @@ static int kvm_vm_ioctl_enable_cap_generic(struct kvm *kvm, kvm->manual_dirty_log_protect = cap->args[0]; return 0; #endif + case KVM_CAP_DIRTY_LOG_RING: + return kvm_vm_ioctl_enable_dirty_log_ring(kvm, cap->args[0]); default: return kvm_vm_ioctl_enable_cap(kvm, cap); } @@ -3449,6 +3614,9 @@ static long kvm_vm_ioctl(struct file *filp, case KVM_CHECK_EXTENSION: r = kvm_vm_ioctl_check_extension_generic(kvm, arg); break; + case KVM_RESET_DIRTY_RINGS: + r = kvm_vm_ioctl_reset_dirty_pages(kvm); + break; default: r = kvm_arch_vm_ioctl(filp, ioctl, arg); } From patchwork Fri Dec 20 21:03:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 11306313 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9EF0E139A for ; Fri, 20 Dec 2019 21:03:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7AD0E2146E for ; Fri, 20 Dec 2019 21:03:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XP6WYOIc" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727811AbfLTVDu (ORCPT ); Fri, 20 Dec 2019 16:03:50 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:30018 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727802AbfLTVDr (ORCPT ); Fri, 20 Dec 2019 16:03:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576875826; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=G5ujQPtmC0gFSTE7FNk323KNrK3wygZRxxl448+O6gM=; b=XP6WYOIcYG02xC7R99/Bg73BCNwd7JieEaaClQWeiU5e3DQPTh0BGUSKNK26Os7c7b9lkz 88p14JRoHTQPHsb0AtBwv1DdRDIcR4L9hO6XIvGqYhMHdbHm9SZXAYRyD1meptEr4qWwE8 qOCiXh1rJHdiB/wZLV01gI6bri5Oa6E= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-65-jCJUZU5fNweUJOJmxBmv1g-1; Fri, 20 Dec 2019 16:03:45 -0500 X-MC-Unique: jCJUZU5fNweUJOJmxBmv1g-1 Received: by mail-qv1-f70.google.com with SMTP id v3so941714qvm.2 for ; Fri, 20 Dec 2019 13:03:45 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=G5ujQPtmC0gFSTE7FNk323KNrK3wygZRxxl448+O6gM=; b=sjxYhdhldA50B2EKfC3v8+S4uRBPePcIvLg6fIb0iN4q2vPg0L3Dm8pBZwHQZTDJvY NI7k0MiTWAIK6qndvsic4WUqj9Qzipf46D/Sd9FBGngJF7wygWVuHzng6TX1cHSLwhvs 2RLUYcvaSDl7fKCE0qyREa51iV/ESUnE1zJ/bfdos8NJ5V9JDx9MkdjC8Jg/KlXpkBhE G0+AaH8iSzPhRjwBtbMzs46LPgAVOJSAH4Dz3FZLN+VZJUrbflckPqzeYO1W4aOXZzs5 ZFxD7DJTxMNFd5fU05dBq9Vo+Xz066lUFfCOJ4HwYEZFkE6EheSQdqhfM6F3iN3DAmHk LmUA== X-Gm-Message-State: APjAAAUaTlHdtvNKPH+wq7plg1fHwNtIkawelbJOCUXY7R4eJGQTj9RJ yY5zd5yCMn1unxjgQPSeUMefbhs7x4bIxfGHE+vObkx/OB5ulTh2ffsl9LtPueGmF3ScNkkf6aN ImU+wCK0+wJv7 X-Received: by 2002:ad4:580e:: with SMTP id dd14mr14365294qvb.84.1576875824950; Fri, 20 Dec 2019 13:03:44 -0800 (PST) X-Google-Smtp-Source: APXvYqyxPVpnfwVCOu4LfWvbWHlZuEFIEhIBc/T7743Ln11mThk5m+SLhgRM0jfbTzEg6gxWfurRpQ== X-Received: by 2002:ad4:580e:: with SMTP id dd14mr14365263qvb.84.1576875824724; Fri, 20 Dec 2019 13:03:44 -0800 (PST) Received: from xz-x1.redhat.com ([104.156.64.74]) by smtp.gmail.com with ESMTPSA id a9sm3061018qtb.36.2019.12.20.13.03.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2019 13:03:44 -0800 (PST) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterx@redhat.com, Sean Christopherson , "Michael S . Tsirkin" , Vitaly Kuznetsov , Paolo Bonzini , Christophe de Dinechin , "Dr . David Alan Gilbert" , Jason Wang Subject: [PATCH v2 09/17] KVM: Make dirty ring exclusive to dirty bitmap log Date: Fri, 20 Dec 2019 16:03:18 -0500 Message-Id: <20191220210326.49949-10-peterx@redhat.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20191220210326.49949-1-peterx@redhat.com> References: <20191220210326.49949-1-peterx@redhat.com> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org There's no good reason to use both the dirty bitmap logging and the new dirty ring buffer to track dirty bits. We should be able to even support both of them at the same time, but it could complicate things which could actually help little. Let's simply make it the rule before we enable dirty ring on any arch, that we don't allow these two interfaces to be used together. The big world switch would be KVM_CAP_DIRTY_LOG_RING capability enablement. That's where we'll switch from the default dirty logging way to the dirty ring way. As long as kvm->dirty_ring_size is setup correctly, we'll once and for all switch to the dirty ring buffer mode for the current virtual machine. Signed-off-by: Peter Xu --- Documentation/virt/kvm/api.txt | 7 +++++++ virt/kvm/kvm_main.c | 12 ++++++++++++ 2 files changed, 19 insertions(+) diff --git a/Documentation/virt/kvm/api.txt b/Documentation/virt/kvm/api.txt index c141b285e673..b507b966f9f1 100644 --- a/Documentation/virt/kvm/api.txt +++ b/Documentation/virt/kvm/api.txt @@ -5411,3 +5411,10 @@ all the existing dirty gfns are flushed to the dirty rings. If one of the ring buffers is full, the guest will exit to userspace with the exit reason set to KVM_EXIT_DIRTY_LOG_FULL, and the KVM_RUN ioctl will return to userspace with zero. + +NOTE: the KVM_CAP_DIRTY_LOG_RING capability and the new ioctl +KVM_RESET_DIRTY_RINGS are exclusive to the existing KVM_GET_DIRTY_LOG +interface. After enabling KVM_CAP_DIRTY_LOG_RING with an acceptable +dirty ring size, the virtual machine will switch to the dirty ring +tracking mode, and KVM_GET_DIRTY_LOG, KVM_CLEAR_DIRTY_LOG ioctls will +stop working. diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 4050631d05f3..b69d34425f8d 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1204,6 +1204,10 @@ int kvm_get_dirty_log(struct kvm *kvm, unsigned long n; unsigned long any = 0; + /* Dirty ring tracking is exclusive to dirty log tracking */ + if (kvm->dirty_ring_size) + return -EINVAL; + as_id = log->slot >> 16; id = (u16)log->slot; if (as_id >= KVM_ADDRESS_SPACE_NUM || id >= KVM_USER_MEM_SLOTS) @@ -1261,6 +1265,10 @@ int kvm_get_dirty_log_protect(struct kvm *kvm, unsigned long *dirty_bitmap; unsigned long *dirty_bitmap_buffer; + /* Dirty ring tracking is exclusive to dirty log tracking */ + if (kvm->dirty_ring_size) + return -EINVAL; + as_id = log->slot >> 16; id = (u16)log->slot; if (as_id >= KVM_ADDRESS_SPACE_NUM || id >= KVM_USER_MEM_SLOTS) @@ -1332,6 +1340,10 @@ int kvm_clear_dirty_log_protect(struct kvm *kvm, unsigned long *dirty_bitmap; unsigned long *dirty_bitmap_buffer; + /* Dirty ring tracking is exclusive to dirty log tracking */ + if (kvm->dirty_ring_size) + return -EINVAL; + as_id = log->slot >> 16; id = (u16)log->slot; if (as_id >= KVM_ADDRESS_SPACE_NUM || id >= KVM_USER_MEM_SLOTS)