From patchwork Sun Nov 4 17:11:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ahmed Soliman X-Patchwork-Id: 10666959 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 22CE715E9 for ; Sun, 4 Nov 2018 17:13:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 051E3289F4 for ; Sun, 4 Nov 2018 17:13:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E71C228A57; Sun, 4 Nov 2018 17:13:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id 35512289F4 for ; Sun, 4 Nov 2018 17:13:28 +0000 (UTC) Received: (qmail 32102 invoked by uid 550); 4 Nov 2018 17:12:42 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 32043 invoked from network); 4 Nov 2018 17:12:41 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=sR5YS7tQy4Qw75ViaeMp3L8IWcGln7wfkKWaGHuSFWE=; b=lWr3sWKvjDoe6kMeJ4m78W+MlXwBaSDUFscSg/Dvb8EJI1bPmZgsUjsSdhj9EJAbzb AYad6ZUh+jDYyC0cn/fppgT+XuVBabxE+R/YlwGFxY/NqFd/tV0Pz8MvTLEsDZJJ/2YH Rghomo+uFV84haI4CO03nlsHZiPVD9NzjyX+/ryck6JO6Yl8HQfXGbUen5tm8BSkCVlP hhY8OKOslkb2EJ+ufUPBFG1kropyaHiTPf0l/AWEG3pkxk+IcQbex57Yl761nYJgrWbW RaUD4wvJeHMUvRbgpoMe/vSGNjMADcTpqgyZjLybrTadBFnw5R3kMWHiibloR6Cp44wN fy/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=sR5YS7tQy4Qw75ViaeMp3L8IWcGln7wfkKWaGHuSFWE=; b=d/Wh89agM1Dh9FMfVKt8jI7N9CcPqdVwRFLxStQH2W54SO/xWEAELovyNoQwKQVi3g C9X1ka1bW36n7xrjKiMBPMy2XWRGBjtdoAtoCZtoT+w2uViz5ckDRvx4jevh89fY46U5 qmB6l6ETVAbNV+8GlVSsb13dCSVeEsen3i+Vp87osrmLBmnO/HNZLNPlRH9J64dSz3aW LYye3ssiQK4mJ4RDQa05TfzThcHbw9rpGWz7W1GI4brYDrHEA0Fju3Vuh0m5HUo36n+V RTUb9cB9Jx3j+Qw3o2whyRZnA48eXW1pPtfN+OBGoltjNZu/rgbjL5Qhl+VQ4ugtJQuj KS4g== X-Gm-Message-State: AGRZ1gJq8seuQUDLi1Ldlz1cgBgPB9CSnZnnhvoF7WziSeqmnqmEwBIu cX7FiZVS6BLYBz10OSfu02U= X-Google-Smtp-Source: AJdET5ffi2FN/aVTU8/FOaMS/7YVSc1g4lQPwCFz06TTueMqm65307GNr/wWiGePLuQ3Z0l7vEh41g== X-Received: by 2002:adf:dbc3:: with SMTP id e3-v6mr17610461wrj.165.1541351550075; Sun, 04 Nov 2018 09:12:30 -0800 (PST) From: Ahmed Abd El Mawgood To: Paolo Bonzini , rkrcmar@redhat.com, Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , hpa@zytor.com, x86@kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, ahmedsoliman0x666@gmail.com, ovich00@gmail.com, kernel-hardening@lists.openwall.com, nigel.edwards@hpe.com, Boris Lukashev , Hossam Hassan <7ossam9063@gmail.com>, "Ahmed Lotfy igor . stoppa @ gmail . com" Subject: [PATCH V6 6/8] KVM: Add support for byte granular memory ROE Date: Sun, 4 Nov 2018 19:11:22 +0200 Message-Id: <20181104171124.5717-7-ahmedsoliman0x666@gmail.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> References: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> X-Virus-Scanned: ClamAV using ClamSMTP This patch documents and implements ROE_MPROTECT_CHUNK, a part of ROE hypercall designed to protect regions of a memory page with byte granularity. This feature provides a key primitive to protect against attacks involving pages remapping. Signed-off-by: Ahmed Abd El Mawgood --- Documentation/virtual/kvm/hypercalls.txt | 9 + include/linux/kvm_host.h | 26 +++ include/uapi/linux/kvm_para.h | 1 + virt/kvm/kvm_main.c | 22 ++- virt/kvm/roe.c | 212 +++++++++++++++++++++-- virt/kvm/roe_generic.h | 15 ++ 6 files changed, 271 insertions(+), 14 deletions(-) diff --git a/Documentation/virtual/kvm/hypercalls.txt b/Documentation/virtual/kvm/hypercalls.txt index 8af64d826f03..a31f316ce6e6 100644 --- a/Documentation/virtual/kvm/hypercalls.txt +++ b/Documentation/virtual/kvm/hypercalls.txt @@ -164,6 +164,15 @@ This configuration lets a guest kernel have part of its read/write memory converted into read-only. This action is irreversible. Upon successful run, the number of pages protected is returned. +Usage 3: + a0: ROE_MPROTECT_CHUNK (requires version >= 2) + a1: Start address aligned to page boundary. + a2: Number of bytes to be protected. +This configuration lets a guest kernel have part of its read/write memory +converted into read-only with bytes granularity. ROE_MPROTECT_CHUNK is +relatively slow compared to ROE_MPROTECT. This action is irreversible. +Upon successful run, the number of bytes protected is returned. + Error codes: -KVM_ENOSYS: system call being triggered from ring 3 or it is not implemented. diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index be6885bc28bc..a6749a52386b 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -294,11 +294,37 @@ static inline int kvm_vcpu_exiting_guest_mode(struct kvm_vcpu *vcpu) */ #define KVM_MEM_MAX_NR_PAGES ((1UL << 31) - 1) +#ifdef CONFIG_KVM_ROE +/* + * This structure is used to hold memory areas that are to be protected in a + * memory frame with mixed page permissions. + **/ +struct protected_chunk { + gpa_t gpa; + u64 size; + struct list_head list; +}; + +static inline bool kvm_roe_range_overlap(struct protected_chunk *chunk, + gpa_t gpa, int len) { + /* + * https://stackoverflow.com/questions/325933/ + * determine-whether-two-date-ranges-overlap + * Assuming that it works, that link ^ provides a solution that is + * better than anything I would ever come up with. + */ + return (gpa <= chunk->gpa + chunk->size - 1) && + (gpa + len - 1 >= chunk->gpa); +} +#endif + struct kvm_memory_slot { gfn_t base_gfn; unsigned long npages; #ifdef CONFIG_KVM_ROE unsigned long *roe_bitmap; + unsigned long *partial_roe_bitmap; + struct list_head *prot_list; #endif unsigned long *dirty_bitmap; struct kvm_arch_memory_slot arch; diff --git a/include/uapi/linux/kvm_para.h b/include/uapi/linux/kvm_para.h index e6004e0750fd..4a84f974bc58 100644 --- a/include/uapi/linux/kvm_para.h +++ b/include/uapi/linux/kvm_para.h @@ -33,6 +33,7 @@ /* ROE Functionality parameters */ #define ROE_VERSION 0 #define ROE_MPROTECT 1 +#define ROE_MPROTECT_CHUNK 2 /* * hypercalls use architecture specific */ diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 814ee0fd3578..48c5d9d9474e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1279,18 +1279,19 @@ static bool memslot_is_readonly(struct kvm_memory_slot *slot) static bool gfn_is_readonly(struct kvm_memory_slot *slot, gfn_t gfn) { - return gfn_is_full_roe(slot, gfn) || memslot_is_readonly(slot); + return gfn_is_full_roe(slot, gfn) || + gfn_is_partial_roe(slot, gfn) || + memslot_is_readonly(slot); } + static unsigned long __gfn_to_hva_many(struct kvm_memory_slot *slot, gfn_t gfn, gfn_t *nr_pages, bool write) { if (!slot || slot->flags & KVM_MEMSLOT_INVALID) return KVM_HVA_ERR_BAD; - if (gfn_is_readonly(slot, gfn) && write) return KVM_HVA_ERR_RO_BAD; - if (nr_pages) *nr_pages = slot->npages - (gfn - slot->base_gfn); @@ -1852,14 +1853,27 @@ int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu, gpa_t gpa, return __kvm_read_guest_atomic(slot, gfn, data, offset, len); } EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_atomic); +static u64 roe_gfn_to_hva(struct kvm_memory_slot *slot, gfn_t gfn, int offset, + int len) +{ + u64 addr; + if (kvm_roe_check_range(slot, gfn, offset, len)) + return KVM_HVA_ERR_RO_BAD; + if (memslot_is_readonly(slot)) + return KVM_HVA_ERR_RO_BAD; + if (gfn_is_full_roe(slot, gfn)) + return KVM_HVA_ERR_RO_BAD; + addr = __gfn_to_hva_many(slot, gfn, NULL, false); + return addr; +} static int __kvm_write_guest_page(struct kvm_memory_slot *memslot, gfn_t gfn, const void *data, int offset, int len) { int r; unsigned long addr; - addr = gfn_to_hva_memslot(memslot, gfn); + addr = roe_gfn_to_hva(memslot, gfn, offset, len); if (kvm_is_error_hva(addr)) return -EFAULT; r = __copy_to_user((void __user *)addr + offset, data, len); diff --git a/virt/kvm/roe.c b/virt/kvm/roe.c index 1a0e4be0198c..e94314fed3a3 100644 --- a/virt/kvm/roe.c +++ b/virt/kvm/roe.c @@ -11,34 +11,89 @@ #include #include #include +#include "roe_generic.h" int kvm_roe_init(struct kvm_memory_slot *slot) { slot->roe_bitmap = kvzalloc(BITS_TO_LONGS(slot->npages) * sizeof(unsigned long), GFP_KERNEL); if (!slot->roe_bitmap) - return -ENOMEM; + goto fail1; + slot->partial_roe_bitmap = kvzalloc(BITS_TO_LONGS(slot->npages) * + sizeof(unsigned long), GFP_KERNEL); + if (!slot->partial_roe_bitmap) + goto fail2; + slot->prot_list = kvzalloc(sizeof(struct list_head), GFP_KERNEL); + if (!slot->prot_list) + goto fail3; + INIT_LIST_HEAD(slot->prot_list); return 0; +fail3: + kvfree(slot->partial_roe_bitmap); +fail2: + kvfree(slot->roe_bitmap); +fail1: + return -ENOMEM; + +} + +static bool kvm_roe_protected_range(struct kvm_memory_slot *slot, gpa_t gpa, + int len) +{ + struct list_head *pos; + struct protected_chunk *cur_chunk; + + list_for_each(pos, slot->prot_list) { + cur_chunk = list_entry(pos, struct protected_chunk, list); + if (kvm_roe_range_overlap(cur_chunk, gpa, len)) + return true; + } + return false; +} + +bool kvm_roe_check_range(struct kvm_memory_slot *slot, gfn_t gfn, int offset, + int len) +{ + gpa_t gpa = (gfn << PAGE_SHIFT) + offset; + if (!gfn_is_partial_roe(slot, gfn)) + return false; + return kvm_roe_protected_range(slot, gpa, len); } + void kvm_roe_free(struct kvm_memory_slot *slot) { + struct protected_chunk *pos, *n; + struct list_head *head = slot->prot_list; + kvfree(slot->roe_bitmap); + kvfree(slot->partial_roe_bitmap); + list_for_each_entry_safe(pos, n, head, list) { + list_del(&pos->list); + kvfree(pos); + } + kvfree(slot->prot_list); } static void kvm_roe_protect_slot(struct kvm *kvm, struct kvm_memory_slot *slot, - gfn_t gfn, u64 npages) + gfn_t gfn, u64 npages, bool partial) { int i; + void *bitmap; + if (partial) + bitmap = slot->partial_roe_bitmap; + else + bitmap = slot->roe_bitmap; for (i = gfn - slot->base_gfn; i < gfn + npages - slot->base_gfn; i++) - set_bit(i, slot->roe_bitmap); + set_bit(i, bitmap); kvm_roe_arch_commit_protection(kvm, slot); } -static int __kvm_roe_protect_range(struct kvm *kvm, gpa_t gpa, u64 npages) +static int __kvm_roe_protect_range(struct kvm *kvm, gpa_t gpa, u64 npages, + bool partial) { struct kvm_memory_slot *slot; gfn_t gfn = gpa >> PAGE_SHIFT; @@ -54,12 +109,12 @@ static int __kvm_roe_protect_range(struct kvm *kvm, gpa_t gpa, u64 npages) if (gfn + npages > slot->base_gfn + slot->npages) { u64 _npages = slot->base_gfn + slot->npages - gfn; - kvm_roe_protect_slot(kvm, slot, gfn, _npages); + kvm_roe_protect_slot(kvm, slot, gfn, _npages, partial); gfn += _npages; count += _npages; npages -= _npages; } else { - kvm_roe_protect_slot(kvm, slot, gfn, npages); + kvm_roe_protect_slot(kvm, slot, gfn, npages, partial); count += npages; npages = 0; } @@ -69,12 +124,13 @@ static int __kvm_roe_protect_range(struct kvm *kvm, gpa_t gpa, u64 npages) return count; } -static int kvm_roe_protect_range(struct kvm *kvm, gpa_t gpa, u64 npages) +static int kvm_roe_protect_range(struct kvm *kvm, gpa_t gpa, u64 npages, + bool partial) { int r; mutex_lock(&kvm->slots_lock); - r = __kvm_roe_protect_range(kvm, gpa, npages); + r = __kvm_roe_protect_range(kvm, gpa, npages, partial); mutex_unlock(&kvm->slots_lock); return r; } @@ -103,7 +159,7 @@ static int kvm_roe_full_protect_range(struct kvm_vcpu *vcpu, u64 gva, continue; if (!access_ok(VERIFY_WRITE, hva, 1 << PAGE_SHIFT)) continue; - status = kvm_roe_protect_range(vcpu->kvm, gpa, 1); + status = kvm_roe_protect_range(vcpu->kvm, gpa, 1, false); if (status > 0) count += status; } @@ -112,6 +168,139 @@ static int kvm_roe_full_protect_range(struct kvm_vcpu *vcpu, u64 gva, return count; } +static int kvm_roe_insert_chunk_next(struct list_head *pos, u64 gpa, u64 size) +{ + struct protected_chunk *chunk; + + chunk = kvzalloc(sizeof(struct protected_chunk), GFP_KERNEL); + chunk->gpa = gpa; + chunk->size = size; + INIT_LIST_HEAD(&chunk->list); + list_add(&chunk->list, pos); + return size; +} + +static int kvm_roe_expand_chunk(struct protected_chunk *pos, u64 gpa, u64 size) +{ + u64 old_ptr = pos->gpa; + u64 old_size = pos->size; + + if (gpa < old_ptr) + pos->gpa = gpa; + if (gpa + size > old_ptr + old_size) + pos->size = gpa + size - pos->gpa; + return size; +} + +static bool kvm_roe_merge_chunks(struct protected_chunk *chunk) +{ + /*attempt merging 2 consecutive given the first one*/ + struct protected_chunk *next = list_next_entry(chunk, list); + + if (!kvm_roe_range_overlap(chunk, next->gpa, next->size)) + return false; + kvm_roe_expand_chunk(chunk, next->gpa, next->size); + list_del(&next->list); + kvfree(next); + return true; +} + +static int __kvm_roe_insert_chunk(struct kvm_memory_slot *slot, u64 gpa, + u64 size) +{ + /* kvm->slots_lock must be acquired*/ + struct protected_chunk *pos; + struct list_head *head = slot->prot_list; + + if (list_empty(head)) + return kvm_roe_insert_chunk_next(head, gpa, size); + /* + * pos here will never get deleted maybe the next one will + * that is why list_for_each_entry_safe is completely unsafe + */ + list_for_each_entry(pos, head, list) { + if (kvm_roe_range_overlap(pos, gpa, size)) { + int ret = kvm_roe_expand_chunk(pos, gpa, size); + + while (head != pos->list.next) + if (!kvm_roe_merge_chunks(pos)) + break; + return ret; + } + if (pos->gpa > gpa) { + struct protected_chunk *prev; + + prev = list_prev_entry(pos, list); + return kvm_roe_insert_chunk_next(&prev->list, gpa, + size); + } + } + pos = list_last_entry(head, struct protected_chunk, list); + + return kvm_roe_insert_chunk_next(&pos->list, gpa, size); +} + +static int kvm_roe_insert_chunk(struct kvm *kvm, u64 gpa, u64 size) +{ + struct kvm_memory_slot *slot; + gfn_t gfn = gpa >> PAGE_SHIFT; + int ret; + + mutex_lock(&kvm->slots_lock); + slot = gfn_to_memslot(kvm, gfn); + ret = __kvm_roe_insert_chunk(slot, gpa, size); + mutex_unlock(&kvm->slots_lock); + return ret; +} + +static int kvm_roe_partial_page_protect(struct kvm_vcpu *vcpu, u64 gva, + u64 size) +{ + gpa_t gpa = kvm_mmu_gva_to_gpa_system(vcpu, gva, NULL); + + kvm_roe_protect_range(vcpu->kvm, gpa, 1, true); + return kvm_roe_insert_chunk(vcpu->kvm, gpa, size); +} + +static int kvm_roe_partial_protect(struct kvm_vcpu *vcpu, u64 gva, u64 size) +{ + u64 gva_start = gva; + u64 gva_end = gva+size; + u64 gpn_start = gva_start >> PAGE_SHIFT; + u64 gpn_end = gva_end >> PAGE_SHIFT; + u64 _size; + int count = 0; + // We need to make sure that there will be no overflow or zero size + if (gva_end <= gva_start) + return -EINVAL; + + // protect the partial page at the start + if (gpn_end > gpn_start) + _size = PAGE_SIZE - (gva_start & PAGE_MASK) + 1; + else + _size = size; + size -= _size; + count += kvm_roe_partial_page_protect(vcpu, gva_start, _size); + // full protect in the middle pages + if (gpn_end - gpn_start > 1) { + int ret; + u64 _gva = (gpn_start + 1) << PAGE_SHIFT; + u64 npages = gpn_end - gpn_start - 1; + + size -= npages << PAGE_SHIFT; + ret = kvm_roe_full_protect_range(vcpu, _gva, npages); + if (ret > 0) + count += ret << PAGE_SHIFT; + } + // protect the partial page at the end + if (size != 0) + count += kvm_roe_partial_page_protect(vcpu, + gpn_end << PAGE_SHIFT, size); + if (count == 0) + return -EINVAL; + return count; +} + int kvm_roe(struct kvm_vcpu *vcpu, u64 a0, u64 a1, u64 a2, u64 a3) { int ret; @@ -123,11 +312,14 @@ int kvm_roe(struct kvm_vcpu *vcpu, u64 a0, u64 a1, u64 a2, u64 a3) return -KVM_ENOSYS; switch (a0) { case ROE_VERSION: - ret = 1; //current version + ret = 2; //current version break; case ROE_MPROTECT: ret = kvm_roe_full_protect_range(vcpu, a1, a2); break; + case ROE_MPROTECT_CHUNK: + ret = kvm_roe_partial_protect(vcpu, a1, a2); + break; default: ret = -EINVAL; } diff --git a/virt/kvm/roe_generic.h b/virt/kvm/roe_generic.h index 42d6ad3a4971..006fc7b52bba 100644 --- a/virt/kvm/roe_generic.h +++ b/virt/kvm/roe_generic.h @@ -13,10 +13,16 @@ void kvm_roe_free(struct kvm_memory_slot *slot); int kvm_roe_init(struct kvm_memory_slot *slot); +bool kvm_roe_check_range(struct kvm_memory_slot *slot, gfn_t gfn, int offset, + int len); static inline bool gfn_is_full_roe(struct kvm_memory_slot *slot, gfn_t gfn) { return test_bit(gfn - slot->base_gfn, slot->roe_bitmap); } +static inline bool gfn_is_partial_roe(struct kvm_memory_slot *slot, gfn_t gfn) +{ + return test_bit(gfn - slot->base_gfn, slot->partial_roe_bitmap); +} #else static void kvm_roe_free(struct kvm_memory_slot *slot) {} static int kvm_roe_init(struct kvm_memory_slot *slot) { return 0; } @@ -24,6 +30,15 @@ static inline bool gfn_is_full_roe(struct kvm_memory_slot *slot, gfn_t gfn) { return false; } +static inline bool gfn_is_partial_roe(struct kvm_memory_slot *slot, gfn_t gfn) +{ + return false; +} +static bool kvm_roe_check_range(struct kvm_memory_slot *slot, gfn_t gfn, + int offset, int len) +{ + return false; +} #endif #endif