From patchwork Sat Jan 11 01:04:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13935691 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2ACA61C27 for ; Sat, 11 Jan 2025 01:04:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736557455; cv=none; b=ProfQ5jP7NzhehbiFJnmTx01/vtO7w4HrtEBcSolfPpFvThVyDGDAs1obbMfgEezPpLRkZe51UJGlV/iZPUOSOX69VoVWr+HqCxrbTxQxvNTaKNBVZQL8ms6KXPPEMU+BFk57UKUavz8ZhpULpqidZ+cWCpfD8IpYCzx529buLs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736557455; c=relaxed/simple; bh=Y87IV5Xpnh+jgd2FxKTBjhTRZsxpUL28iWLdqHeOOJE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=oKNCu60ePVUhKwB8vf4GWHvDAaoUzQ3BjQmTBS73diEKUVFYba1MBO4zXG3x1BqG6XiulYo1UuCQ7oDp2+ZI6BuFkjLGKcNfNlbkafgX6xQ3wPFudzhS2nYGxbe6LJxlNiv+sv3EKPbDGsUZZAGOb+4jMcWwKsDLG9Ayie4bJ+o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SLqjOoSr; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SLqjOoSr" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2ef9864e006so7429444a91.2 for ; Fri, 10 Jan 2025 17:04:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736557453; x=1737162253; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Cy2pnqQ2fSzBgMiJrF6O6bTMaX9oe7O9BUSjAJGBR+0=; b=SLqjOoSryRv1XsjDEu9pvYFmixZ+y5ks07rjgwvUxIAntrzq17skWikXjZE7jSr3rE 7KJAobL/YPvRbgXyjcVI1S9LG9eaOpITDpC2HEwB0N27EHMW7o8Fj8QVLKoUizjcxKpL hc4EXRWI9BDwFyW0IXJGVF7lACrfncT6xOJN7iuMHnBm/nBMTHtXpxqQEkfFQV3lHxyS qD6k7ENdlurjmXlEotNJA9SGISGvAcSGSO4D0C0rOFvY1H2vohHxbVp56gW87UbDsiF0 zom9usmNp3OnXkTAe8aAV9UfKFut00bcYUBRqrgOC3tMlZ/I/UDFOWG7I2rgBpWGOWTz NVfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736557453; x=1737162253; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Cy2pnqQ2fSzBgMiJrF6O6bTMaX9oe7O9BUSjAJGBR+0=; b=few5auy4VbYwhSB9xYXyJDNiGJ3eQs2rXU3ucathUIKD25sEK0zUwQnKeet4RM77TQ Jcty9ME6ZVl0HnXC/+Ef3Xo0PozoL6/RERAlmqiDP4isNQyUk0/eJASBEJr2aLKep7K3 uCtwDtTe603NNSx96qbv1oZXonwa6rXV2b+f+N7NIqIO/pyJH6fajXn0QjfbX2nseBDR Hj2yuSONja6+kKXGbOtf0gkwzImSvzRMnbpm+p+Zjx0E9qrfCG0LXu0sEVVRutZZSP5z J7UQnjN5ZopGBnhy0kM9yCL/CKLF9TpZm1d5vK6t3aAju7OwP4sOXLaDfyyadohHBh/u zkHg== X-Gm-Message-State: AOJu0YwI3Mhf7yyDfsy2fZhbV1dZvBj9gFJljM9MKpXyMEEMiVLRNFPw Grt91DgUeGTw60E0uiGZuAJyCBR1NbzMMRu9w1JyBOOUMVvNysdlBuDhWjalX/1Gfme560ULMLY Ukw== X-Google-Smtp-Source: AGHT+IEiLT8u+IwJHWKAAuUTYAyOIXDxsf9AxyDDoSlDQKImRmj4TEJpD11oowKHMJ/OBwLbI6iHkHS2p5s= X-Received: from pfbln9.prod.google.com ([2002:a05:6a00:3cc9:b0:72a:f9c7:a2ed]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:408e:b0:725:9dc7:4f8b with SMTP id d2e1a72fcca58-72d21f562c6mr17589432b3a.15.1736557453546; Fri, 10 Jan 2025 17:04:13 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 10 Jan 2025 17:04:05 -0800 In-Reply-To: <20250111010409.1252942-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250111010409.1252942-1-seanjc@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20250111010409.1252942-2-seanjc@google.com> Subject: [PATCH 1/5] KVM: Bound the number of dirty ring entries in a single reset at INT_MAX From: Sean Christopherson To: Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Xu , Yan Zhao , Maxim Levitsky , Sean Christopherson Cap the number of ring entries that are reset in a single ioctl to INT_MAX to ensure userspace isn't confused by a wrap into negative space, and so that, in a truly pathological scenario, KVM doesn't miss a TLB flush due to the count wrapping to zero. While the size of the ring is fixed at 0x10000 entries and KVM (currently) supports at most 4096, userspace is allowed to harvest entries from the ring while the reset is in-progress, i.e. it's possible for the ring to always harvested entries. Opportunistically return an actual error code from the helper so that a future fix to handle pending signals can gracefully return -EINTR. Cc: Peter Xu Cc: Yan Zhao Cc: Maxim Levitsky Fixes: fb04a1eddb1a ("KVM: X86: Implement ring-based dirty memory tracking") Signed-off-by: Sean Christopherson --- include/linux/kvm_dirty_ring.h | 8 +++++--- virt/kvm/dirty_ring.c | 10 +++++----- virt/kvm/kvm_main.c | 9 ++++++--- 3 files changed, 16 insertions(+), 11 deletions(-) diff --git a/include/linux/kvm_dirty_ring.h b/include/linux/kvm_dirty_ring.h index 4862c98d80d3..82829243029d 100644 --- a/include/linux/kvm_dirty_ring.h +++ b/include/linux/kvm_dirty_ring.h @@ -49,9 +49,10 @@ static inline int kvm_dirty_ring_alloc(struct kvm_dirty_ring *ring, } static inline int kvm_dirty_ring_reset(struct kvm *kvm, - struct kvm_dirty_ring *ring) + struct kvm_dirty_ring *ring, + int *nr_entries_reset) { - return 0; + return -ENOENT; } static inline void kvm_dirty_ring_push(struct kvm_vcpu *vcpu, @@ -81,7 +82,8 @@ int kvm_dirty_ring_alloc(struct kvm_dirty_ring *ring, int index, u32 size); * called with kvm->slots_lock held, returns the number of * processed pages. */ -int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring); +int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring, + int *nr_entries_reset); /* * returns =0: successfully pushed diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c index 7bc74969a819..2faf894dec5a 100644 --- a/virt/kvm/dirty_ring.c +++ b/virt/kvm/dirty_ring.c @@ -104,19 +104,19 @@ static inline bool kvm_dirty_gfn_harvested(struct kvm_dirty_gfn *gfn) return smp_load_acquire(&gfn->flags) & KVM_DIRTY_GFN_F_RESET; } -int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring) +int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring, + int *nr_entries_reset) { u32 cur_slot, next_slot; u64 cur_offset, next_offset; unsigned long mask; - int count = 0; struct kvm_dirty_gfn *entry; bool first_round = true; /* This is only needed to make compilers happy */ cur_slot = cur_offset = mask = 0; - while (true) { + while (likely((*nr_entries_reset) < INT_MAX)) { entry = &ring->dirty_gfns[ring->reset_index & (ring->size - 1)]; if (!kvm_dirty_gfn_harvested(entry)) @@ -129,7 +129,7 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring) kvm_dirty_gfn_set_invalid(entry); ring->reset_index++; - count++; + (*nr_entries_reset)++; /* * Try to coalesce the reset operations when the guest is * scanning pages in the same slot. @@ -166,7 +166,7 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring) trace_kvm_dirty_ring_reset(ring); - return count; + return 0; } void kvm_dirty_ring_push(struct kvm_vcpu *vcpu, u32 slot, u64 offset) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 9d54473d18e3..2d63b4d46ccb 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -4877,15 +4877,18 @@ static int kvm_vm_ioctl_reset_dirty_pages(struct kvm *kvm) { unsigned long i; struct kvm_vcpu *vcpu; - int cleared = 0; + int cleared = 0, r; if (!kvm->dirty_ring_size) return -EINVAL; mutex_lock(&kvm->slots_lock); - kvm_for_each_vcpu(i, vcpu, kvm) - cleared += kvm_dirty_ring_reset(vcpu->kvm, &vcpu->dirty_ring); + kvm_for_each_vcpu(i, vcpu, kvm) { + r = kvm_dirty_ring_reset(vcpu->kvm, &vcpu->dirty_ring, &cleared); + if (r) + break; + } mutex_unlock(&kvm->slots_lock); From patchwork Sat Jan 11 01:04:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13935692 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05F8921106 for ; Sat, 11 Jan 2025 01:04:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736557457; cv=none; b=DmUrzV3Qdnl90ZWaHPtKlqtrvkY30XOmJUP2xe95Qlq3mQhSzrIWyguUv2Xp1rgWVrl1K1EtVCrWYqmuZNzEsnt/Jy+rjFkO4aE7CNbPqjRTy8SREuajdXhbw6F+YIdxVl/nkykq79Woo2s6UnvqLDqm5BpYbkQgPU3dq0KcHIw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736557457; c=relaxed/simple; bh=xIq2EpqRyA2AXMVcTKTfGnLziwxJNZwMY0SpruaP6dQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fHgcBV8gpZ+YhEEMzq9zI5a/IZ64ZrTV65ZR+cEzFloR3/xfDUD7BFKXrY4COATYVrOTLXD+hx4VkgxhF/EwcqCu6icqPfbyQ+AGioxcMif3/Ft6XOJtJYZCMrjNf7A0G5g7IkPRLzwnVXLYrXxOMiiP4v6gXSNWO80IKIRqjE0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=4qs5oJlp; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="4qs5oJlp" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ee46799961so6714186a91.2 for ; Fri, 10 Jan 2025 17:04:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736557455; x=1737162255; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=qLONhXo3aW3WBH6fEqaAtaxz8PcaBZClYQxjCE2fW5k=; b=4qs5oJlp1NssvfORJ4Fz8n0E7/9Q9YF6XBJ3KFSFnXvaaAWsBWw1SVWB7aW1eRxyck Wn65+z/O5Lf/+namTAQj9Lb95a2TFGMpZNE+B93P4SwWgV9s1ytC2WTpktBQTUrDjuWu MRDtshiYsP7ay+wgdPVxE/xxIpU6kTv6pzGbAo2N+Ys1aETlClp+FqZrchyifISTaZiZ /H+lxtQByZMBz6nbxJ2OFA69N9NanXwdSR8CdNFKfjuT3GqGAVhtI7/HD3MzZsFzNqsQ TWBQ2RB7ojnF3/PIp7AR5JEFoKOc4mjiiCKKwq9gasQ1lBQgF0R0448AUghuwSc+uVEP YSGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736557455; x=1737162255; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=qLONhXo3aW3WBH6fEqaAtaxz8PcaBZClYQxjCE2fW5k=; b=QD6zzbbPrlWCv0V+K3QhacsNScOGfLQxcXyYVefAq07XaYq4L4gl8VuZXLSfz25HqW /nh5ZzRp/hb5o+Y8oNjT1qvfonAMREztlHmRYLai9knUePGtNX4+eCXjBuD9AItA6nZ0 QEEPKJHy3JCmj+xK4ZXC8EdPNOby5ry+LN2Iyra9bsspicAEeSWXLD6RP1PK80bhdoqQ 8SF4saEbOey42K/z0CmIq1mN90CTs3daDf8oatIiOqMyXxuhmpQ6N4O6vKCeVzdfAv7R kxUFwWw4wtTaN7NFpzjn+TMi6VNCgt4C5f2zJtfiD0RcyzzBkW1HgXuHnTVUunImihSP eY/Q== X-Gm-Message-State: AOJu0YwjKw7LQyJWZWT+1990oIN3Bx4Uglej1nfYC+Gn/T1QMiaBn5FJ Pl4BxIJCGO+IX6VyPbfrHD0ZCmEdAR7hUA52slsrmJFP5DWRCCnBzxnsqT/FlNqFIFUDr9eViGU koQ== X-Google-Smtp-Source: AGHT+IHH0Fmds1PmmVVZr37IVwKQmveAAVRjINELYisNKzj4RspyqLxQGG7qVO+G/iH8KfvqVTQcx1AsZ7I= X-Received: from pjbqi17.prod.google.com ([2002:a17:90b:2751:b0:2ea:5084:5297]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2b8e:b0:2ee:d433:7c54 with SMTP id 98e67ed59e1d1-2f548eceae7mr17545829a91.19.1736557455348; Fri, 10 Jan 2025 17:04:15 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 10 Jan 2025 17:04:06 -0800 In-Reply-To: <20250111010409.1252942-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250111010409.1252942-1-seanjc@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20250111010409.1252942-3-seanjc@google.com> Subject: [PATCH 2/5] KVM: Bail from the dirty ring reset flow if a signal is pending From: Sean Christopherson To: Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Xu , Yan Zhao , Maxim Levitsky , Sean Christopherson Abort a dirty ring reset if the current task has a pending signal, as the hard limit of INT_MAX entries doesn't ensure KVM will respond to a signal in a timely fashion. Fixes: fb04a1eddb1a ("KVM: X86: Implement ring-based dirty memory tracking") Signed-off-by: Sean Christopherson --- virt/kvm/dirty_ring.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c index 2faf894dec5a..a81ad17d5eef 100644 --- a/virt/kvm/dirty_ring.c +++ b/virt/kvm/dirty_ring.c @@ -117,6 +117,9 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring, cur_slot = cur_offset = mask = 0; while (likely((*nr_entries_reset) < INT_MAX)) { + if (signal_pending(current)) + return -EINTR; + entry = &ring->dirty_gfns[ring->reset_index & (ring->size - 1)]; if (!kvm_dirty_gfn_harvested(entry)) From patchwork Sat Jan 11 01:04:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13935693 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9880B1854 for ; Sat, 11 Jan 2025 01:04:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736557460; cv=none; b=KGof17pU+2Ys877XmYctrf7zT5ozCS4MATVg584u5tkcPPp7/Db70UOIr+Jrv3vAnN9WhD+zJhmZl0qAd7O+BZXCVFKGwUAf8qHnFZ39bAIkJYB6DIiVQ+3gPd2MlmeNgHtDkX7XZ0l1gaSj5HtdJHYL9SGCxlfsMjlhMM6aYfs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736557460; c=relaxed/simple; bh=38yKh50jmTM2QnQCwQHA2pD1K0ArDB2ne9BY9y0yDsE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XSOSunP+ypgTTaYfv1QIkqAC7yqIZOEyEuBr67QejNfDIPRUAa+oluALJZLV3TzcTzk7WeZ2P0w7J1sWrq9zwMo6cKRMPgzuaCt+EDwowyF9WRKNKxjY6Y92cy9uayf7ByH7Xcf/ofvmvQMQuLT0HQdb+mzNqPFJaoBTmOH8Hn4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=OtwLxFBh; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="OtwLxFBh" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2efc3292021so6672102a91.1 for ; Fri, 10 Jan 2025 17:04:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736557457; x=1737162257; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=qztCq3yN4c7V5VVWefuCXzG4z+KIAvAw68bTrwIGEiM=; b=OtwLxFBhXTN+tJ+2KUpBBqdXmvLd6QivviqhooqBIdpPjzAQnYEbmbLWXTQ9OKKAhZ h0PMoIdDfPxwSd4KEOmmwsQXCGZmb+s+5LUFFIqPq0kmHUK3gKlMT/7oZtwcS2OMmdCu fJKGsKXxyoCtAxaspOiRLZ95rwhrWXwyFL9Rr3QO8LLMcLC1oekmaPZ3Jse/NZmc4gFP qCAKEV0Q+pAU76dvkHUW/p7t++udkCGmsHuthpVWKbm1PfqhORHzxQyRORavYKTC3LU3 +A4WvGYoqIkrAYcMm6tiMEW+Uw04OReJlLPcrpSiJWcP4I/8eX5xNBVU1lxKO5MPXJ/G 9Ocg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736557457; x=1737162257; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=qztCq3yN4c7V5VVWefuCXzG4z+KIAvAw68bTrwIGEiM=; b=Z3UjRNEwX9y7gcgAKMFQkvZTowoYqG45Evkauy5RkDGjkyJ3DXjyXTbNIOhqGbHuRt u6965ai+95aeRNm5RfCXXR/T80REaclGyavOt9Jt4yMyQ2jhDdEa+8pzer+8oPC6nno0 Udy9ao64CMrWXc71r3IK1jV218K6L7d2yDIxWwVkGE5UCpw1lvaN+GEOMnbJdOKyvWSO M4Ix1pMdI84xSvxYmneJi0ik9YXBZ1dmWSbJ0EkBuigwDvpcfJnlmyz5AlAqmdLIdfr9 zCLkk2QssyI20giBWSRqdwVW19NZUvPfwNivV01O2R2AGO9fArBMbb6sPzMu3vUbW4iv rLJA== X-Gm-Message-State: AOJu0YzEs+POfMt/Yu0S/MTdsF2ElcraunkAtnKM8Mn210+Ne0CW2zxc xaeXvXXJJdyXID875lhBivGvYVZ/Ed51w21z4xg70t3Bx46p9hjJsmU1orL/YNY+EyGk68UtPC6 d4w== X-Google-Smtp-Source: AGHT+IGWC2DYlQkU/EcPOOmwCPjaBsLhvhJQuYZtuDOFD2C754CM0kGzgUC90dVegSd+WxYgpA7SBB0o1w0= X-Received: from pjc4.prod.google.com ([2002:a17:90b:2f44:b0:2f2:ea3f:34c3]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:274e:b0:2f4:4003:f3ea with SMTP id 98e67ed59e1d1-2f5490f19c7mr19708701a91.33.1736557457046; Fri, 10 Jan 2025 17:04:17 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 10 Jan 2025 17:04:07 -0800 In-Reply-To: <20250111010409.1252942-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250111010409.1252942-1-seanjc@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20250111010409.1252942-4-seanjc@google.com> Subject: [PATCH 3/5] KVM: Conditionally reschedule when resetting the dirty ring From: Sean Christopherson To: Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Xu , Yan Zhao , Maxim Levitsky , Sean Christopherson When resetting a dirty ring, conditionally reschedule on each iteration after the first. The recently introduced hard limit mitigates the issue of an endless reset, but isn't sufficient to completely prevent RCU stalls, soft lockups, etc., nor is the hard limit intended to guard against such badness. Note! Take care to check for reschedule even in the "continue" paths, as a pathological scenario (or malicious userspace) could dirty the same gfn over and over, i.e. always hit the continue path. rcu: INFO: rcu_sched self-detected stall on CPU rcu: 4-....: (5249 ticks this GP) idle=51e4/1/0x4000000000000000 softirq=309/309 fqs=2563 rcu: (t=5250 jiffies g=-319 q=608 ncpus=24) CPU: 4 UID: 1000 PID: 1067 Comm: dirty_log_test Tainted: G L 6.13.0-rc3-17fa7a24ea1e-HEAD-vm #814 Tainted: [L]=SOFTLOCKUP Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015 RIP: 0010:kvm_arch_mmu_enable_log_dirty_pt_masked+0x26/0x200 [kvm] Call Trace: kvm_reset_dirty_gfn.part.0+0xb4/0xe0 [kvm] kvm_dirty_ring_reset+0x58/0x220 [kvm] kvm_vm_ioctl+0x10eb/0x15d0 [kvm] __x64_sys_ioctl+0x8b/0xb0 do_syscall_64+0x5b/0x160 entry_SYSCALL_64_after_hwframe+0x4b/0x53 Tainted: [L]=SOFTLOCKUP Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015 RIP: 0010:kvm_arch_mmu_enable_log_dirty_pt_masked+0x17/0x200 [kvm] Call Trace: kvm_reset_dirty_gfn.part.0+0xb4/0xe0 [kvm] kvm_dirty_ring_reset+0x58/0x220 [kvm] kvm_vm_ioctl+0x10eb/0x15d0 [kvm] __x64_sys_ioctl+0x8b/0xb0 do_syscall_64+0x5b/0x160 entry_SYSCALL_64_after_hwframe+0x4b/0x53 Fixes: fb04a1eddb1a ("KVM: X86: Implement ring-based dirty memory tracking") Signed-off-by: Sean Christopherson --- virt/kvm/dirty_ring.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c index a81ad17d5eef..37eb2b7142bd 100644 --- a/virt/kvm/dirty_ring.c +++ b/virt/kvm/dirty_ring.c @@ -133,6 +133,16 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring, ring->reset_index++; (*nr_entries_reset)++; + + /* + * While the size of each ring is fixed, it's possible for the + * ring to be constantly re-dirtied/harvested while the reset + * is in-progress (the hard limit exists only to guard against + * wrapping the count into negative space). + */ + if (!first_round) + cond_resched(); + /* * Try to coalesce the reset operations when the guest is * scanning pages in the same slot. From patchwork Sat Jan 11 01:04:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13935694 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC8E614830F for ; Sat, 11 Jan 2025 01:04:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736557462; cv=none; b=IX9a/qTLFmv/1+Z1sK8qmJjlTY9+CeejsBY7Z4HzUymzghWMTRm15+V7yiQl+u7R7S1hB3Pz/iyb8omVUsf4YkPn1qr4ftDbrvSHpDxYyBzdf5oumXQovW+pJcE5ufyj97I+eN3t32/aCAyttSvsW+h9j98piVYFK+UGUqahnf8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736557462; c=relaxed/simple; bh=4AOojhK7RgZpfHVFyZZg/F1b+66In+n0W7FklHWokKs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PqpOQRIPps2YbsoKfwBCLd33VoM6bc/MCR0YFwxQ9Cbzk83h2MI/R6K5GUXH98I9FVqiIECWSmPCQJbCNrbaoV1fGMNtu11+e00XBm5NLn5EhPNGsTV0rmiIqojCkDQrM7LRxx8kiNoVKOyg+Vh6SjpUr6f4cbUbETAZb7wFeKA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SS4Dk6LS; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SS4Dk6LS" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2eebfd6d065so6710519a91.3 for ; Fri, 10 Jan 2025 17:04:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736557458; x=1737162258; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=oGi7slrEyYAnwuVNpr6ZrhtkCDMVeK/CUwkbMoeq/cE=; b=SS4Dk6LSfD/sZVtJuIMAVMzMhrSGOGVePoGpUZYs9dQbksms/LORsyTEjwcBrCiPsm Zx7QTXq1sVxb/+vcWoutAybzoOisOCfZ9MnwaZYq7SP6twNqXyf522SM4BIbVdZ4z3MN zf3yj1qjBK4Z60Fv/IoxZNu1gzAzsK6Z4AilPcbZV58NKEHEZD4j284X7kW1wcRw1Ttt 6Wb4NMMzHwag04ebjlaDTguwP/Vn4VYkMM90SojZYjHg12YeMX40p5/zAlwK3rPyFxqy rC8f3Jsl5EbxgU+E/U03ZGZcvwyyz7W09BBQCIG4+Q9r8CPciGEpcPS0lzKsPbCWK1fe E7PA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736557458; x=1737162258; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=oGi7slrEyYAnwuVNpr6ZrhtkCDMVeK/CUwkbMoeq/cE=; b=m8ZOfqODANiRgxq32bO87+CMG+qvZdD7mpL6h1FHa6MghWyFD3mV13tWseEw+1y+Uz PxQvAWTNDcfjVYQJgJdTS039SBy36PNZiA2hZI1YWUMVgf16Iw4/koqmMyhe1QvQmDuT VgBfxaghdOjpZ2tEFnr1aMBqy0/4nB2B9o0ae0TaC5OaXvbQD7vEAYRc5+9s4iMbsdF8 +sOJY7v7+uhOb7+ehI6XwUT9PbcQQpYYd/JSlgblXf96MznImTZ8HprIRAqHENCTMlvG jJfES5d4WvLRpcLOzhL7o7UBYDBPGNiifiutu+JJOlvOQsSrrR5vPj6sg/G/b+BPALnx iPjg== X-Gm-Message-State: AOJu0YzME1FuFhC87Uvu8/xZcnIpJS6eIsA0RnSsYMFVKTSRvShkRYBr BIHOoInn9oqx4GFacdkng/hlxAyGJqJg00Rh94EtvN4PctU8JMZi8D/0OrCCX7TuIQk30NW5fZN fQA== X-Google-Smtp-Source: AGHT+IEU0JMvjhrXcRzpNyyxrvcQqODJ1wZPu5PG2qjeWRbJZlS0/3S5feFaFOwVGoqKiMB61Iex5Pfl42k= X-Received: from pfwz22.prod.google.com ([2002:a05:6a00:1d96:b0:725:e37e:7451]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:3e13:b0:725:eacf:cfda with SMTP id d2e1a72fcca58-72d21fe0263mr18801122b3a.17.1736557458526; Fri, 10 Jan 2025 17:04:18 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 10 Jan 2025 17:04:08 -0800 In-Reply-To: <20250111010409.1252942-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250111010409.1252942-1-seanjc@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20250111010409.1252942-5-seanjc@google.com> Subject: [PATCH 4/5] KVM: Check for empty mask of harvested dirty ring entries in caller From: Sean Christopherson To: Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Xu , Yan Zhao , Maxim Levitsky , Sean Christopherson When resetting a dirty ring, explicitly check that there is work to be done before calling kvm_reset_dirty_gfn(), e.g. if no harvested entries are found and/or on the loop's first iteration, and delete the extremely misleading comment "This is only needed to make compilers happy". KVM absolutely relies on mask to be zero-initialized, i.e. the comment is an outright lie. Furthermore, the compiler is right to complain that KVM is calling a function with uninitialized data, as there are no guarantees the implementation details of kvm_reset_dirty_gfn() will be visible to kvm_dirty_ring_reset(). While the flaw could be fixed by simply deleting (or rewording) the comment, and duplicating the check is unfortunate, checking mask in the caller will allow for additional cleanups. Opportunisticaly drop the zero-initialization of cur_slot and cur_offset. If a bug were introduced where either the slot or offset was consumed before mask is set to a non-zero value, then it is highly desirable for the compiler (or some other sanitizer) to yell. Cc: Peter Xu Cc: Yan Zhao Cc: Maxim Levitsky Signed-off-by: Sean Christopherson --- virt/kvm/dirty_ring.c | 29 ++++++++++++++++++++--------- 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c index 37eb2b7142bd..95ab0e3cf9da 100644 --- a/virt/kvm/dirty_ring.c +++ b/virt/kvm/dirty_ring.c @@ -55,9 +55,6 @@ static void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask) struct kvm_memory_slot *memslot; int as_id, id; - if (!mask) - return; - as_id = slot >> 16; id = (u16)slot; @@ -109,13 +106,10 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring, { u32 cur_slot, next_slot; u64 cur_offset, next_offset; - unsigned long mask; + unsigned long mask = 0; struct kvm_dirty_gfn *entry; bool first_round = true; - /* This is only needed to make compilers happy */ - cur_slot = cur_offset = mask = 0; - while (likely((*nr_entries_reset) < INT_MAX)) { if (signal_pending(current)) return -EINTR; @@ -163,14 +157,31 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring, continue; } } - kvm_reset_dirty_gfn(kvm, cur_slot, cur_offset, mask); + + /* + * Reset the slot for all the harvested entries that have been + * gathered, but not yet fully processed. + */ + if (mask) + kvm_reset_dirty_gfn(kvm, cur_slot, cur_offset, mask); + + /* + * The current slot was reset or this is the first harvested + * entry, (re)initialize the metadata. + */ cur_slot = next_slot; cur_offset = next_offset; mask = 1; first_round = false; } - kvm_reset_dirty_gfn(kvm, cur_slot, cur_offset, mask); + /* + * Perform a final reset if there are harvested entries that haven't + * been processed. The loop only performs a reset when an entry can't + * be coalesced, i.e. always leaves at least one entry pending. + */ + if (mask) + kvm_reset_dirty_gfn(kvm, cur_slot, cur_offset, mask); /* * The request KVM_REQ_DIRTY_RING_SOFT_FULL will be cleared From patchwork Sat Jan 11 01:04:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13935695 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1BD01487D5 for ; Sat, 11 Jan 2025 01:04:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736557462; cv=none; b=Zn2Mh9tZP6nQbMZC7PBH/h6zGg3AqO4xcXZerhdjawTWcpLe9LjQMh1Gok3ncaPcdU27aBfMClnF9qE+GEemuIAhglM2lOnl6rUAiLaNfezvL38zeKvJYZStXZxsdnunTVxy+AqCYxhORz0va8RXolbMYqWRCzCRU6Wo2x4VS28= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736557462; c=relaxed/simple; bh=GuHaUoJc3kSQRY79m3B6lcu5nI5McGM/FGvOckPh/E4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Wrvl06nqcjBD/nuko6+m9+LhAxeAoJzixGHm5V49hwzz4eplC0lbJ1/KLa2oiF0F5UIiNFCy5oO7cBcEQc0iaxthB8sutE/96laQtdY43lJM7qFjGbb0Vpbxj1WfDHG2lyn8kk5XBD8MS0oHjeiceA+rM2TNgFODG+6CnsJHu9Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=A6ZI9l3A; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="A6ZI9l3A" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2eeeb5b7022so4804483a91.0 for ; Fri, 10 Jan 2025 17:04:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736557460; x=1737162260; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=tVYbCX2RqPz2ig5eCEENys/M+ZhhahZJXVbEX0SbPgo=; b=A6ZI9l3AManGIewequpKdftymqIyiAKM8gSb7o4oxl+z8KJ5jPV3Pn3YUM6CTJejSW zizpj1S01JQnas30amkG2ucciGB2wdF5Vcb3YwlCDE+m3aAPjzVqpO/8NEnmGVDfLs98 m+sKTIl82YnGSpYwFKBbA7xmTtPVGO7XTpVV5TwThk90I5ie3oOCNOV5VqAbXmZJOHyI 1u0ytJZLS1flfT7YwzwJHXvD9EQKn/EZyUQqx+pVJ8lEYlOibV0nzNXqHmwrplY8g7IB GUUmpZ0mPLE30uAya38B/FngIOH9e+Jq7PaZi1ZQnCUghwkZP15kOfwkZpE5Z+gK5kD1 0DHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736557460; x=1737162260; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=tVYbCX2RqPz2ig5eCEENys/M+ZhhahZJXVbEX0SbPgo=; b=I8O3QmUsxtEmRCMx6YQ8wz6JX9/DFt1GMwaQ7S8zYSM7EVSArESg4zYNGD97g0ynyj jqyDaJ+y3jS/dLCyl6OUUU29qa+PKtmVln8QTr1cVjfs0xwaWf9FgcOyUIvtov3KTvdX o+PKG/2CfO7xR2MIfc55Lj4u25SiZW1BiLf7UMqlzHtM0zjwocL+6eFAIdVMst/rXhCr 0R5nsOfDglp2nsYV7SjUgvigWmkMsBIu79LPWerrHWjqPCEMlz6X92u4dbIfg5avE8Ow F0g1qlTWvTCXvHMjG7IODv2r7x5ZsIn37Abder95jGtDcwC8ZS0fuCXzAo9MGqR9sU0E 5NdQ== X-Gm-Message-State: AOJu0Ywa6CtnVSVUd9V0/Q7WdsVZYLITemXyXQmRB0iOjU+P/ifbnWQv Ryv9r7UIMLKp0sex7XgbPa2a7U3T+9XmJhK+4HY2Bx3PkyrzLDWleEovLiNCffsdqkJuJnoRAA7 FSg== X-Google-Smtp-Source: AGHT+IFuOLVPsL+7D49WGbUBlqnRDugnmoXi0Idkmj5v+pibXtrs67/9IO5GA1O0JQ4pwUp/7WG4rrkB8Ok= X-Received: from pjbsw3.prod.google.com ([2002:a17:90b:2c83:b0:2f4:3ea1:9033]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:c2c7:b0:2ee:e518:c1d8 with SMTP id 98e67ed59e1d1-2f548f1c3f0mr18718339a91.30.1736557460283; Fri, 10 Jan 2025 17:04:20 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 10 Jan 2025 17:04:09 -0800 In-Reply-To: <20250111010409.1252942-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250111010409.1252942-1-seanjc@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20250111010409.1252942-6-seanjc@google.com> Subject: [PATCH 5/5] KVM: Use mask of harvested dirty ring entries to coalesce dirty ring resets From: Sean Christopherson To: Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Peter Xu , Yan Zhao , Maxim Levitsky , Sean Christopherson Use "mask" instead of a dedicated boolean to track whether or not there is at least one to-be-reset entry for the current slot+offset. In the body of the loop, mask is zero only on the first iteration, i.e. !mask is equivalent to first_round. Opportunstically combine the adjacent "if (mask)" statements into a single if-statement. No function change intended. Cc: Peter Xu Cc: Yan Zhao Cc: Maxim Levitsky Signed-off-by: Sean Christopherson --- virt/kvm/dirty_ring.c | 60 +++++++++++++++++++++---------------------- 1 file changed, 29 insertions(+), 31 deletions(-) diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c index 95ab0e3cf9da..9b23f86ff7b6 100644 --- a/virt/kvm/dirty_ring.c +++ b/virt/kvm/dirty_ring.c @@ -108,7 +108,6 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring, u64 cur_offset, next_offset; unsigned long mask = 0; struct kvm_dirty_gfn *entry; - bool first_round = true; while (likely((*nr_entries_reset) < INT_MAX)) { if (signal_pending(current)) @@ -128,42 +127,42 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring, ring->reset_index++; (*nr_entries_reset)++; - /* - * While the size of each ring is fixed, it's possible for the - * ring to be constantly re-dirtied/harvested while the reset - * is in-progress (the hard limit exists only to guard against - * wrapping the count into negative space). - */ - if (!first_round) + if (mask) { + /* + * While the size of each ring is fixed, it's possible + * for the ring to be constantly re-dirtied/harvested + * while the reset is in-progress (the hard limit exists + * only to guard against the count becoming negative). + */ cond_resched(); - /* - * Try to coalesce the reset operations when the guest is - * scanning pages in the same slot. - */ - if (!first_round && next_slot == cur_slot) { - s64 delta = next_offset - cur_offset; + /* + * Try to coalesce the reset operations when the guest + * is scanning pages in the same slot. + */ + if (next_slot == cur_slot) { + s64 delta = next_offset - cur_offset; - if (delta >= 0 && delta < BITS_PER_LONG) { - mask |= 1ull << delta; - continue; - } + if (delta >= 0 && delta < BITS_PER_LONG) { + mask |= 1ull << delta; + continue; + } - /* Backwards visit, careful about overflows! */ - if (delta > -BITS_PER_LONG && delta < 0 && - (mask << -delta >> -delta) == mask) { - cur_offset = next_offset; - mask = (mask << -delta) | 1; - continue; + /* Backwards visit, careful about overflows! */ + if (delta > -BITS_PER_LONG && delta < 0 && + (mask << -delta >> -delta) == mask) { + cur_offset = next_offset; + mask = (mask << -delta) | 1; + continue; + } } - } - /* - * Reset the slot for all the harvested entries that have been - * gathered, but not yet fully processed. - */ - if (mask) + /* + * Reset the slot for all the harvested entries that + * have been gathered, but not yet fully processed. + */ kvm_reset_dirty_gfn(kvm, cur_slot, cur_offset, mask); + } /* * The current slot was reset or this is the first harvested @@ -172,7 +171,6 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring, cur_slot = next_slot; cur_offset = next_offset; mask = 1; - first_round = false; } /*