From patchwork Fri Jun 6 17:33:41 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mario Smarduch X-Patchwork-Id: 4313151 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id F329A9F170 for ; Fri, 6 Jun 2014 17:37:28 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id D20FF20171 for ; Fri, 6 Jun 2014 17:37:27 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B266020166 for ; Fri, 6 Jun 2014 17:37:26 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1Wsy22-00020p-4p; Fri, 06 Jun 2014 17:34:22 +0000 Received: from mailout1.w2.samsung.com ([211.189.100.11] helo=usmailout1.samsung.com) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1Wsy1y-000200-R0 for linux-arm-kernel@lists.infradead.org; Fri, 06 Jun 2014 17:34:20 +0000 Received: from uscpsbgex2.samsung.com (u123.gpu85.samsung.co.kr [203.254.195.123]) by mailout1.w2.samsung.com (Oracle Communications Messaging Server 7u4-24.01(7.0.4.24.0) 64bit (built Nov 17 2011)) with ESMTP id <0N6R006Y5CS8JU30@mailout1.w2.samsung.com> for linux-arm-kernel@lists.infradead.org; Fri, 06 Jun 2014 13:33:44 -0400 (EDT) X-AuditID: cbfec37b-b7fbe6d000006742-27-5391fb785bbe Received: from usmmp2.samsung.com ( [203.254.195.78]) by uscpsbgex2.samsung.com (USCPEXMTA) with SMTP id B1.36.26434.87BF1935; Fri, 06 Jun 2014 13:33:44 -0400 (EDT) Received: from sisasmtp.sisa.samsung.com ([105.144.21.116]) by usmmp2.samsung.com (Oracle Communications Messaging Server 7u4-27.01(7.0.4.27.0) 64bit (built Aug 30 2012)) with ESMTP id <0N6R0028JCS8X5D0@usmmp2.samsung.com>; Fri, 06 Jun 2014 13:33:44 -0400 (EDT) Received: from mjsmard-530U3C-530U4C-532U3C.sisa.samsung.com (105.144.129.76) by SISAEX02SJ.sisa.samsung.com (105.144.21.116) with Microsoft SMTP Server (TLS) id 14.1.421.2; Fri, 06 Jun 2014 10:33:43 -0700 From: Mario Smarduch To: kvmarm@lists.cs.columbia.edu, marc.zyngier@arm.com, christoffer.dall@linaro.org Subject: [RESEND PATCH v7 3/4] arm: dirty log write protect management support Date: Fri, 06 Jun 2014 10:33:41 -0700 Message-id: <1402076021-9425-1-git-send-email-m.smarduch@samsung.com> X-Mailer: git-send-email 1.7.9.5 MIME-version: 1.0 X-Originating-IP: [105.144.129.76] X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrELMWRmVeSWpSXmKPExsVy+t9hP92K3xODDf5v1bJ48fofo0Xv/4us Fvevfme0mDO10OLjqePsFpseX2O1+HvnH5vFnDMPWCwmvdnGZPFhxkpGi4X/bzI6cHusmbeG 0WNWQy+bx51re9g8HhzazOJxftMaZo/NS+o9+rasYvT4vEkugCOKyyYlNSezLLVI3y6BK2Pv k2amgnOBFX9PNDI3MH506GLk5JAQMJHYvPIgO4QtJnHh3nq2LkYuDiGBZYwSfRdPsUA4vUwS U5ZdhspcZJR48mE2C0gLm4CuxP57G8HaRQRCJa7/bWQCKWIW6GSS+LfrMVhCWMBP4tvNW6wg NouAqsST7sNsIDavgKvEgf43QHEOoN0KEnMm2UCEBSV+TL7HAhJmFpCQeP5ZCSQsBNS57eZz RohqJYnVR8wnMArMQtIwC6FhASPTKkax0uLkguKk9NQKI73ixNzi0rx0veT83E2MkHio3sF4 96vNIUYBDkYlHl6LjxODhVgTy4orcw8xSnAwK4nwsjUDhXhTEiurUovy44tKc1KLDzEycXBK NTDmFPbvldx+bdWcOt75u58ef5hq/OiAy2KVsrPN83/dZX7BJBZdddtFb1NvxYIHP/5fVln0 5uK0faKu5xw8ci+Zb1iZ9Wvq8TWyF6aqpvf73F61I6ZPrHvlEs37pQcqnl3cb6rTnpl5r6nj w9J7b+5dU7IQF7E72nfOZueOCVfas7KLrd7VbGTMV2Ipzkg01GIuKk4EADd0gKdlAgAA X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20140606_103419_035117_6D3AAC78 X-CRM114-Status: GOOD ( 23.64 ) X-Spam-Score: -5.6 (-----) Cc: peter.maydell@linaro.org, kvm@vger.kernel.org, steve.capper@arm.com, xiaoguangrong@linux.vnet.ibm.com, linux-arm-kernel@lists.infradead.org, jays.lee@samsung.com, sungjinn.chung@samsung.com, gavin.guo@canonical.com, Mario Smarduch X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP kvm_vm_ioctl_get_dirty_log() is generic used by x86, ARM. x86 recent patch changed this function, this patch picks up those changes, re-tested everything works. Applies cleanly with other patches. This patch adds support for keeping track of VM dirty pages. As dirty page log is retrieved, the pages that have been written are write protected again for next write and log read. Signed-off-by: Mario Smarduch --- arch/arm/include/asm/kvm_host.h | 3 ++ arch/arm/kvm/arm.c | 5 --- arch/arm/kvm/mmu.c | 79 +++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 86 --------------------------------------- virt/kvm/kvm_main.c | 86 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 168 insertions(+), 91 deletions(-) diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h index 59565f5..b760f9c 100644 --- a/arch/arm/include/asm/kvm_host.h +++ b/arch/arm/include/asm/kvm_host.h @@ -232,5 +232,8 @@ u64 kvm_arm_timer_get_reg(struct kvm_vcpu *, u64 regid); int kvm_arm_timer_set_reg(struct kvm_vcpu *, u64 regid, u64 value); void kvm_mmu_wp_memory_region(struct kvm *kvm, int slot); +void kvm_mmu_write_protect_pt_masked(struct kvm *kvm, + struct kvm_memory_slot *slot, + gfn_t gfn_offset, unsigned long mask); #endif /* __ARM_KVM_HOST_H__ */ diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c index dfd63ac..f06fb21 100644 --- a/arch/arm/kvm/arm.c +++ b/arch/arm/kvm/arm.c @@ -780,11 +780,6 @@ long kvm_arch_vcpu_ioctl(struct file *filp, } } -int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log) -{ - return -EINVAL; -} - static int kvm_vm_ioctl_set_device_addr(struct kvm *kvm, struct kvm_arm_device_addr *dev_addr) { diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c index e5dff85..907344c 100644 --- a/arch/arm/kvm/mmu.c +++ b/arch/arm/kvm/mmu.c @@ -874,6 +874,85 @@ void kvm_mmu_wp_memory_region(struct kvm *kvm, int slot) spin_unlock(&kvm->mmu_lock); } +/** + * stage2_wp_mask_range() - write protect memslot pages set in mask + * @pmd - pointer to page table + * @start_ipa - the start range of mask + * @addr - start_ipa or start range of adjusted mask if crossing PMD range + * @mask - mask of dirty pages + * + * Walk mask and write protect the associated dirty pages in the memory region. + * If mask crosses a PMD range adjust it to next page table and return. + */ +static void stage2_wp_mask_range(pmd_t *pmd, phys_addr_t start_ipa, + phys_addr_t *addr, unsigned long *mask) +{ + pte_t *pte; + bool crosses_pmd; + int i = __ffs(*mask); + + if (unlikely(*addr > start_ipa)) + start_ipa = *addr - i * PAGE_SIZE; + pte = pte_offset_kernel(pmd, start_ipa); + for (*addr = start_ipa + i * PAGE_SIZE; *mask; + i = __ffs(*mask), *addr = start_ipa + i * PAGE_SIZE) { + crosses_pmd = !!((start_ipa & PMD_MASK) ^ (*addr & PMD_MASK)); + if (unlikely(crosses_pmd)) { + /* Adjust mask dirty bits relative to next page table */ + *mask >>= (PTRS_PER_PTE - pte_index(start_ipa)); + return; + } + if (!pte_none(pte[i])) + kvm_set_s2pte_readonly(&pte[i]); + *mask &= ~(1 << i); + } +} + +/** + * kvm_mmu_write_protected_pt_masked() - write protect dirty pages set in mask + * @kvm: The KVM pointer + * @slot: The memory slot associated with mask + * @gfn_offset: The gfn offset in memory slot + * @mask: The mask of dirty pages at offset 'gnf_offset' in this memory + * slot to be write protected + * + * Called from dirty page logging read function to write protect bits set in + * mask to record future writes to these pages in dirty page log. This function + * uses simplified page table walk given mask can spawn no more then 2 PMD + * table range. + * 'kvm->mmu_lock' must be held to protect against concurrent modification + * of page tables (2nd stage fault, mmu modifiers, ...) + * + */ +void kvm_mmu_write_protect_pt_masked(struct kvm *kvm, + struct kvm_memory_slot *slot, + gfn_t gfn_offset, unsigned long mask) +{ + pud_t *pud; + pmd_t *pmd; + phys_addr_t start_ipa = (slot->base_gfn + gfn_offset) << PAGE_SHIFT; + phys_addr_t end_ipa = start_ipa + BITS_PER_LONG * PAGE_SIZE; + phys_addr_t addr = start_ipa; + pgd_t *pgdp = kvm->arch.pgd, *pgd; + + do { + pgd = pgdp + pgd_index(addr); + if (pgd_present(*pgd)) { + pud = pud_offset(pgd, addr); + if (!pud_none(*pud) && !pud_huge(*pud)) { + pmd = pmd_offset(pud, addr); + if (!pmd_none(*pmd) && !kvm_pmd_huge(*pmd)) + stage2_wp_mask_range(pmd, start_ipa, + &addr, &mask); + else + addr += PMD_SIZE; + } else + addr += PUD_SIZE; + } else + addr += PGDIR_SIZE; + } while (mask && addr < end_ipa); +} + static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, struct kvm_memory_slot *memslot, unsigned long fault_status) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c5582c3..a603ca3 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3569,92 +3569,6 @@ static int kvm_vm_ioctl_reinject(struct kvm *kvm, return 0; } -/** - * kvm_vm_ioctl_get_dirty_log - get and clear the log of dirty pages in a slot - * @kvm: kvm instance - * @log: slot id and address to which we copy the log - * - * We need to keep it in mind that VCPU threads can write to the bitmap - * concurrently. So, to avoid losing data, we keep the following order for - * each bit: - * - * 1. Take a snapshot of the bit and clear it if needed. - * 2. Write protect the corresponding page. - * 3. Flush TLB's if needed. - * 4. Copy the snapshot to the userspace. - * - * Between 2 and 3, the guest may write to the page using the remaining TLB - * entry. This is not a problem because the page will be reported dirty at - * step 4 using the snapshot taken before and step 3 ensures that successive - * writes will be logged for the next call. - */ -int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log) -{ - int r; - struct kvm_memory_slot *memslot; - unsigned long n, i; - unsigned long *dirty_bitmap; - unsigned long *dirty_bitmap_buffer; - bool is_dirty = false; - - mutex_lock(&kvm->slots_lock); - - r = -EINVAL; - if (log->slot >= KVM_USER_MEM_SLOTS) - goto out; - - memslot = id_to_memslot(kvm->memslots, log->slot); - - dirty_bitmap = memslot->dirty_bitmap; - r = -ENOENT; - if (!dirty_bitmap) - goto out; - - n = kvm_dirty_bitmap_bytes(memslot); - - dirty_bitmap_buffer = dirty_bitmap + n / sizeof(long); - memset(dirty_bitmap_buffer, 0, n); - - spin_lock(&kvm->mmu_lock); - - for (i = 0; i < n / sizeof(long); i++) { - unsigned long mask; - gfn_t offset; - - if (!dirty_bitmap[i]) - continue; - - is_dirty = true; - - mask = xchg(&dirty_bitmap[i], 0); - dirty_bitmap_buffer[i] = mask; - - offset = i * BITS_PER_LONG; - kvm_mmu_write_protect_pt_masked(kvm, memslot, offset, mask); - } - - spin_unlock(&kvm->mmu_lock); - - /* See the comments in kvm_mmu_slot_remove_write_access(). */ - lockdep_assert_held(&kvm->slots_lock); - - /* - * All the TLBs can be flushed out of mmu lock, see the comments in - * kvm_mmu_slot_remove_write_access(). - */ - if (is_dirty) - kvm_flush_remote_tlbs(kvm); - - r = -EFAULT; - if (copy_to_user(log->dirty_bitmap, dirty_bitmap_buffer, n)) - goto out; - - r = 0; -out: - mutex_unlock(&kvm->slots_lock); - return r; -} - int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_event, bool line_status) { diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index ba25765..c87c612 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -429,6 +429,92 @@ static int kvm_init_mmu_notifier(struct kvm *kvm) return mmu_notifier_register(&kvm->mmu_notifier, current->mm); } +/** + * kvm_vm_ioctl_get_dirty_log - get and clear the log of dirty pages in a slot + * @kvm: kvm instance + * @log: slot id and address to which we copy the log + * + * We need to keep it in mind that VCPU threads can write to the bitmap + * concurrently. So, to avoid losing data, we keep the following order for + * each bit: + * + * 1. Take a snapshot of the bit and clear it if needed. + * 2. Write protect the corresponding page. + * 3. Flush TLB's if needed. + * 4. Copy the snapshot to the userspace. + * + * Between 2 and 3, the guest may write to the page using the remaining TLB + * entry. This is not a problem because the page will be reported dirty at + * step 4 using the snapshot taken before and step 3 ensures that successive + * writes will be logged for the next call. + */ +int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log) +{ + int r; + struct kvm_memory_slot *memslot; + unsigned long n, i; + unsigned long *dirty_bitmap; + unsigned long *dirty_bitmap_buffer; + bool is_dirty = false; + + mutex_lock(&kvm->slots_lock); + + r = -EINVAL; + if (log->slot >= KVM_USER_MEM_SLOTS) + goto out; + + memslot = id_to_memslot(kvm->memslots, log->slot); + + dirty_bitmap = memslot->dirty_bitmap; + r = -ENOENT; + if (!dirty_bitmap) + goto out; + + n = kvm_dirty_bitmap_bytes(memslot); + + dirty_bitmap_buffer = dirty_bitmap + n / sizeof(long); + memset(dirty_bitmap_buffer, 0, n); + + spin_lock(&kvm->mmu_lock); + + for (i = 0; i < n / sizeof(long); i++) { + unsigned long mask; + gfn_t offset; + + if (!dirty_bitmap[i]) + continue; + + is_dirty = true; + + mask = xchg(&dirty_bitmap[i], 0); + dirty_bitmap_buffer[i] = mask; + + offset = i * BITS_PER_LONG; + kvm_mmu_write_protect_pt_masked(kvm, memslot, offset, mask); + } + + spin_unlock(&kvm->mmu_lock); + + /* See the comments in kvm_mmu_slot_remove_write_access(). */ + lockdep_assert_held(&kvm->slots_lock); + + /* + * All the TLBs can be flushed out of mmu lock, see the comments in + * kvm_mmu_slot_remove_write_access(). + */ + if (is_dirty) + kvm_flush_remote_tlbs(kvm); + + r = -EFAULT; + if (copy_to_user(log->dirty_bitmap, dirty_bitmap_buffer, n)) + goto out; + + r = 0; +out: + mutex_unlock(&kvm->slots_lock); + return r; +} + #else /* !(CONFIG_MMU_NOTIFIER && KVM_ARCH_WANT_MMU_NOTIFIER) */ static int kvm_init_mmu_notifier(struct kvm *kvm)