From patchwork Sun Nov 4 17:11:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ahmed Soliman X-Patchwork-Id: 10666935 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B3E5D13A4 for ; Sun, 4 Nov 2018 17:12:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9AEE0295AE for ; Sun, 4 Nov 2018 17:12:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8F19B295CB; Sun, 4 Nov 2018 17:12:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id C57F0295AE for ; Sun, 4 Nov 2018 17:12:32 +0000 (UTC) Received: (qmail 28471 invoked by uid 550); 4 Nov 2018 17:12:27 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 28416 invoked from network); 4 Nov 2018 17:12:26 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=K83MXMKnKynCqSGpEAGqilub5PPGoz3eEK23HTZ8YhQ=; b=N6FGvY/NYAFUMsnDD39lrWWLMzDPDQefmYJ5naoiet50aA1WqSrX/XiC87EeZffT39 AoSx98S6LP4rPzTSnKsDcHh8LvtW0TIWKY3ut77+680gFsIUvv/GvXAwPnqIMbMgIUdB hMrGE5YgbvJvdOXZdjHLJ18y2Jkx+tGui/E1z1/rm7VZPepjIzecEwXk8+V2SAkB3k9w Wq8f8Aqny9zZ4RjW8JyjDb6uQ9xnx7TTh4L9qcEdqZPRMtqzhppTRkHLFvwKnFGf0qy6 skt/Lft055jd200W9tNnj0yN2mimBBToElCDjc/QR1b5LgJHfyrOVRt/rX/xXZwQAfUo zT9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=K83MXMKnKynCqSGpEAGqilub5PPGoz3eEK23HTZ8YhQ=; b=M1eQBGwCo0iS4wypSUYzVrP2UJWjWUKOQ5k8h/uP70gzExCG1DQve/J5yAljKnWriT bzanTdkJ0pY+5seyLMdNBfUfLs1t5uybeeVgzKZvRjXRsWWJ9sPXRwQKLTWha1FNOyLe c71dVPWluGGVAHz76aqqf4PFpT0eUZEZpHkM1u831Xm6+ASyvDUcePdqZTLFXl/WLUYc LDleXhprgK+2TzsbdGG2k+OdHS0/1I7bN5jGB99YzJiE9TubbcPWyhnsJOrcS60iIixB q9hylY5gJGn2QHh7y+8BjeWIirHgwm7ogfUdJCV6ZSLqo+XGaFlSUYdZsd1aBtU2b2ZW 8mgQ== X-Gm-Message-State: AGRZ1gK/Sjp6ex8pikK2ZRMaAe0SzDNQ/9LneruNPEB2WPOWEUFxXL3A K6KuwVcPAOY5OGctLJJlq7E= X-Google-Smtp-Source: AJdET5cQ3WZTW+AYb94a97tNBct6qOTGljjvw5Vecie00821Zbfei8diLvIUTGbfbdyRro7e8kTPQA== X-Received: by 2002:a1c:2e50:: with SMTP id u77-v6mr3889431wmu.106.1541351534712; Sun, 04 Nov 2018 09:12:14 -0800 (PST) From: Ahmed Abd El Mawgood To: Paolo Bonzini , rkrcmar@redhat.com, Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , hpa@zytor.com, x86@kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, ahmedsoliman0x666@gmail.com, ovich00@gmail.com, kernel-hardening@lists.openwall.com, nigel.edwards@hpe.com, Boris Lukashev , Hossam Hassan <7ossam9063@gmail.com>, "Ahmed Lotfy igor . stoppa @ gmail . com" Subject: [PATCH V6 1/8] KVM: State whether memory should be freed in kvm_free_memslot Date: Sun, 4 Nov 2018 19:11:17 +0200 Message-Id: <20181104171124.5717-2-ahmedsoliman0x666@gmail.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> References: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> X-Virus-Scanned: ClamAV using ClamSMTP The conditions upon which kvm_free_memslot are kind of confusing, it will be hard to extend memslot with allocatable data that needs to be freed, so I replaced the current mechanism by the change flag, it states if the memory slot should be freed or not. Signed-off-by: Ahmed Abd El Mawgood --- virt/kvm/kvm_main.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 2679e476b6c3..039c1ef9a786 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -549,9 +549,10 @@ static void kvm_destroy_dirty_bitmap(struct kvm_memory_slot *memslot) * Free any memory in @free but not in @dont. */ static void kvm_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free, - struct kvm_memory_slot *dont) + struct kvm_memory_slot *dont, + enum kvm_mr_change change) { - if (!dont || free->dirty_bitmap != dont->dirty_bitmap) + if (change == KVM_MR_DELETE) kvm_destroy_dirty_bitmap(free); kvm_arch_free_memslot(kvm, free, dont); @@ -567,7 +568,7 @@ static void kvm_free_memslots(struct kvm *kvm, struct kvm_memslots *slots) return; kvm_for_each_memslot(memslot, slots) - kvm_free_memslot(kvm, memslot, NULL); + kvm_free_memslot(kvm, memslot, NULL, KVM_MR_DELETE); kvfree(slots); } @@ -1063,14 +1064,14 @@ int __kvm_set_memory_region(struct kvm *kvm, kvm_arch_commit_memory_region(kvm, mem, &old, &new, change); - kvm_free_memslot(kvm, &old, &new); + kvm_free_memslot(kvm, &old, &new, change); kvfree(old_memslots); return 0; out_slots: kvfree(slots); out_free: - kvm_free_memslot(kvm, &new, &old); + kvm_free_memslot(kvm, &new, &old, change); out: return r; } From patchwork Sun Nov 4 17:11:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ahmed Soliman X-Patchwork-Id: 10666937 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CCA4513A4 for ; Sun, 4 Nov 2018 17:12:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B274D295AE for ; Sun, 4 Nov 2018 17:12:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A6432295CB; Sun, 4 Nov 2018 17:12:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id 3B532295AE for ; Sun, 4 Nov 2018 17:12:41 +0000 (UTC) Received: (qmail 29841 invoked by uid 550); 4 Nov 2018 17:12:30 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 29770 invoked from network); 4 Nov 2018 17:12:29 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=kF/k/zyomP4gQjXdIQh9uoVods1BUkDTWlT2w7Wn68I=; b=h+8Z7CBiV3f7/NJVCc32ju1C//FkAH7hE5PMK1WiNeHtwILGseoOfZEIVRsZtle5xe J4X4+RpM8edBNYMmWBF+M5EfANupcrCs0KM/eDYHVCQEPmCMwXxrzkHjAb9e5ToOEqJ0 05OxTIl8q3K1wfCXG8gV45hIK57U+rlHIfRv2IXW7aSYlS0oDFWtzXLBgEVzVt9zp5IZ IMZlcVpTguWJcZLfuQu/+Ds04RZo0hPNup16IzDjg9O/1NRfhSj+7uy3eeZaYo2pXU2H 1X9uWAaMv2TXmaZix4QHaA4YBmpwbLHZEH/8XM73OGFqy8uM/Tj8V2HbvJaBd9SBlIY8 QJBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=kF/k/zyomP4gQjXdIQh9uoVods1BUkDTWlT2w7Wn68I=; b=JAiUosQWvwzcJycp7nx7cD+OTu01VUtIcf3S07EkL2tu8SoU9jbg4VUbbYifIv4Q+T zgP7Ttm3KHi7sXKbfVZO1zT5AM76itkQs/v4lfBk1396lrvXOgINDm2EJtQHiULnlEEy 1NkWj74QqNbqd/NgNTV7u3UrvZrMb9cdPM7ct8s32EPOFUAhRkCWQaJFv/z+yQawUDV8 hkZC9pIoFMaOH+TXuTWPCQ1YH8oS9sjcz1LSEBVccA4WU/p0Ok75SlrD5+3Nes+HWUO2 R2WIDziWBXO3IwjwRRkgx3VALVQQN6NgLCIzSKYCxLONxBMxK5bYHGNznZ/VwlikDwVj YB6w== X-Gm-Message-State: AGRZ1gKk+lVUUj46/ZUOy9EddlOcvmqEXiqgVw5sdYFTAlMkJn/aoesF ohpghLIeIa5yNhscQp1Rbh4= X-Google-Smtp-Source: AJdET5dn8W58UZjiu5B6U/gD6+wtoGe3weaBazrQSNxA26X9fX0DFXNIlnjrWkEIiYvhsFyTrq+9nw== X-Received: by 2002:a1c:f417:: with SMTP id z23-v6mr3830121wma.80.1541351537715; Sun, 04 Nov 2018 09:12:17 -0800 (PST) From: Ahmed Abd El Mawgood To: Paolo Bonzini , rkrcmar@redhat.com, Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , hpa@zytor.com, x86@kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, ahmedsoliman0x666@gmail.com, ovich00@gmail.com, kernel-hardening@lists.openwall.com, nigel.edwards@hpe.com, Boris Lukashev , Hossam Hassan <7ossam9063@gmail.com>, "Ahmed Lotfy igor . stoppa @ gmail . com" Subject: [PATCH V6 2/8] KVM: X86: Add arbitrary data pointer in kvm memslot iterator functions Date: Sun, 4 Nov 2018 19:11:18 +0200 Message-Id: <20181104171124.5717-3-ahmedsoliman0x666@gmail.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> References: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> X-Virus-Scanned: ClamAV using ClamSMTP This will help sharing data into the slot_level_handler callback. In my case I need to a share a counter for the pages traversed to use it in some bitmap. Being able to send arbitrary memory pointer into the slot_level_handler callback made it easy. Signed-off-by: Ahmed Abd El Mawgood --- arch/x86/kvm/mmu.c | 65 ++++++++++++++++++++++++++-------------------- 1 file changed, 37 insertions(+), 28 deletions(-) diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index cf5f572f2305..c54ec914935b 100644 --- a/arch/x86/kvm/mmu.c +++ b/arch/x86/kvm/mmu.c @@ -1492,7 +1492,7 @@ static bool spte_write_protect(u64 *sptep, bool pt_protect) static bool __rmap_write_protect(struct kvm *kvm, struct kvm_rmap_head *rmap_head, - bool pt_protect) + bool pt_protect, void *data) { u64 *sptep; struct rmap_iterator iter; @@ -1531,7 +1531,8 @@ static bool wrprot_ad_disabled_spte(u64 *sptep) * - W bit on ad-disabled SPTEs. * Returns true iff any D or W bits were cleared. */ -static bool __rmap_clear_dirty(struct kvm *kvm, struct kvm_rmap_head *rmap_head) +static bool __rmap_clear_dirty(struct kvm *kvm, struct kvm_rmap_head *rmap_head, + void *data) { u64 *sptep; struct rmap_iterator iter; @@ -1557,7 +1558,8 @@ static bool spte_set_dirty(u64 *sptep) return mmu_spte_update(sptep, spte); } -static bool __rmap_set_dirty(struct kvm *kvm, struct kvm_rmap_head *rmap_head) +static bool __rmap_set_dirty(struct kvm *kvm, struct kvm_rmap_head *rmap_head, + void *data) { u64 *sptep; struct rmap_iterator iter; @@ -1589,7 +1591,7 @@ static void kvm_mmu_write_protect_pt_masked(struct kvm *kvm, while (mask) { rmap_head = __gfn_to_rmap(slot->base_gfn + gfn_offset + __ffs(mask), PT_PAGE_TABLE_LEVEL, slot); - __rmap_write_protect(kvm, rmap_head, false); + __rmap_write_protect(kvm, rmap_head, false, NULL); /* clear the first set bit */ mask &= mask - 1; @@ -1615,7 +1617,7 @@ void kvm_mmu_clear_dirty_pt_masked(struct kvm *kvm, while (mask) { rmap_head = __gfn_to_rmap(slot->base_gfn + gfn_offset + __ffs(mask), PT_PAGE_TABLE_LEVEL, slot); - __rmap_clear_dirty(kvm, rmap_head); + __rmap_clear_dirty(kvm, rmap_head, NULL); /* clear the first set bit */ mask &= mask - 1; @@ -1668,7 +1670,8 @@ bool kvm_mmu_slot_gfn_write_protect(struct kvm *kvm, for (i = PT_PAGE_TABLE_LEVEL; i <= PT_MAX_HUGEPAGE_LEVEL; ++i) { rmap_head = __gfn_to_rmap(gfn, i, slot); - write_protected |= __rmap_write_protect(kvm, rmap_head, true); + write_protected |= __rmap_write_protect(kvm, rmap_head, true, + NULL); } return write_protected; @@ -1682,7 +1685,8 @@ static bool rmap_write_protect(struct kvm_vcpu *vcpu, u64 gfn) return kvm_mmu_slot_gfn_write_protect(vcpu->kvm, slot, gfn); } -static bool kvm_zap_rmapp(struct kvm *kvm, struct kvm_rmap_head *rmap_head) +static bool kvm_zap_rmapp(struct kvm *kvm, struct kvm_rmap_head *rmap_head, + void *data) { u64 *sptep; struct rmap_iterator iter; @@ -1702,7 +1706,7 @@ static int kvm_unmap_rmapp(struct kvm *kvm, struct kvm_rmap_head *rmap_head, struct kvm_memory_slot *slot, gfn_t gfn, int level, unsigned long data) { - return kvm_zap_rmapp(kvm, rmap_head); + return kvm_zap_rmapp(kvm, rmap_head, NULL); } static int kvm_set_pte_rmapp(struct kvm *kvm, struct kvm_rmap_head *rmap_head, @@ -5523,13 +5527,15 @@ void kvm_mmu_uninit_vm(struct kvm *kvm) } /* The return value indicates if tlb flush on all vcpus is needed. */ -typedef bool (*slot_level_handler) (struct kvm *kvm, struct kvm_rmap_head *rmap_head); +typedef bool (*slot_level_handler) (struct kvm *kvm, + struct kvm_rmap_head *rmap_head, void *data); /* The caller should hold mmu-lock before calling this function. */ static __always_inline bool slot_handle_level_range(struct kvm *kvm, struct kvm_memory_slot *memslot, slot_level_handler fn, int start_level, int end_level, - gfn_t start_gfn, gfn_t end_gfn, bool lock_flush_tlb) + gfn_t start_gfn, gfn_t end_gfn, bool lock_flush_tlb, + void *data) { struct slot_rmap_walk_iterator iterator; bool flush = false; @@ -5537,7 +5543,7 @@ slot_handle_level_range(struct kvm *kvm, struct kvm_memory_slot *memslot, for_each_slot_rmap_range(memslot, start_level, end_level, start_gfn, end_gfn, &iterator) { if (iterator.rmap) - flush |= fn(kvm, iterator.rmap); + flush |= fn(kvm, iterator.rmap, data); if (need_resched() || spin_needbreak(&kvm->mmu_lock)) { if (flush && lock_flush_tlb) { @@ -5559,36 +5565,36 @@ slot_handle_level_range(struct kvm *kvm, struct kvm_memory_slot *memslot, static __always_inline bool slot_handle_level(struct kvm *kvm, struct kvm_memory_slot *memslot, slot_level_handler fn, int start_level, int end_level, - bool lock_flush_tlb) + bool lock_flush_tlb, void *data) { return slot_handle_level_range(kvm, memslot, fn, start_level, end_level, memslot->base_gfn, memslot->base_gfn + memslot->npages - 1, - lock_flush_tlb); + lock_flush_tlb, data); } static __always_inline bool slot_handle_all_level(struct kvm *kvm, struct kvm_memory_slot *memslot, - slot_level_handler fn, bool lock_flush_tlb) + slot_level_handler fn, bool lock_flush_tlb, void *data) { return slot_handle_level(kvm, memslot, fn, PT_PAGE_TABLE_LEVEL, - PT_MAX_HUGEPAGE_LEVEL, lock_flush_tlb); + PT_MAX_HUGEPAGE_LEVEL, lock_flush_tlb, data); } static __always_inline bool slot_handle_large_level(struct kvm *kvm, struct kvm_memory_slot *memslot, - slot_level_handler fn, bool lock_flush_tlb) + slot_level_handler fn, bool lock_flush_tlb, void *data) { return slot_handle_level(kvm, memslot, fn, PT_PAGE_TABLE_LEVEL + 1, - PT_MAX_HUGEPAGE_LEVEL, lock_flush_tlb); + PT_MAX_HUGEPAGE_LEVEL, lock_flush_tlb, data); } static __always_inline bool slot_handle_leaf(struct kvm *kvm, struct kvm_memory_slot *memslot, - slot_level_handler fn, bool lock_flush_tlb) + slot_level_handler fn, bool lock_flush_tlb, void *data) { return slot_handle_level(kvm, memslot, fn, PT_PAGE_TABLE_LEVEL, - PT_PAGE_TABLE_LEVEL, lock_flush_tlb); + PT_PAGE_TABLE_LEVEL, lock_flush_tlb, data); } void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end) @@ -5610,7 +5616,7 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end) slot_handle_level_range(kvm, memslot, kvm_zap_rmapp, PT_PAGE_TABLE_LEVEL, PT_MAX_HUGEPAGE_LEVEL, - start, end - 1, true); + start, end - 1, true, NULL); } } @@ -5618,9 +5624,10 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end) } static bool slot_rmap_write_protect(struct kvm *kvm, - struct kvm_rmap_head *rmap_head) + struct kvm_rmap_head *rmap_head, + void *data) { - return __rmap_write_protect(kvm, rmap_head, false); + return __rmap_write_protect(kvm, rmap_head, false, data); } void kvm_mmu_slot_remove_write_access(struct kvm *kvm, @@ -5630,7 +5637,7 @@ void kvm_mmu_slot_remove_write_access(struct kvm *kvm, spin_lock(&kvm->mmu_lock); flush = slot_handle_all_level(kvm, memslot, slot_rmap_write_protect, - false); + false, NULL); spin_unlock(&kvm->mmu_lock); /* @@ -5656,7 +5663,8 @@ void kvm_mmu_slot_remove_write_access(struct kvm *kvm, } static bool kvm_mmu_zap_collapsible_spte(struct kvm *kvm, - struct kvm_rmap_head *rmap_head) + struct kvm_rmap_head *rmap_head, + void *data) { u64 *sptep; struct rmap_iterator iter; @@ -5694,7 +5702,7 @@ void kvm_mmu_zap_collapsible_sptes(struct kvm *kvm, /* FIXME: const-ify all uses of struct kvm_memory_slot. */ spin_lock(&kvm->mmu_lock); slot_handle_leaf(kvm, (struct kvm_memory_slot *)memslot, - kvm_mmu_zap_collapsible_spte, true); + kvm_mmu_zap_collapsible_spte, true, NULL); spin_unlock(&kvm->mmu_lock); } @@ -5704,7 +5712,7 @@ void kvm_mmu_slot_leaf_clear_dirty(struct kvm *kvm, bool flush; spin_lock(&kvm->mmu_lock); - flush = slot_handle_leaf(kvm, memslot, __rmap_clear_dirty, false); + flush = slot_handle_leaf(kvm, memslot, __rmap_clear_dirty, false, NULL); spin_unlock(&kvm->mmu_lock); lockdep_assert_held(&kvm->slots_lock); @@ -5727,7 +5735,7 @@ void kvm_mmu_slot_largepage_remove_write_access(struct kvm *kvm, spin_lock(&kvm->mmu_lock); flush = slot_handle_large_level(kvm, memslot, slot_rmap_write_protect, - false); + false, NULL); spin_unlock(&kvm->mmu_lock); /* see kvm_mmu_slot_remove_write_access */ @@ -5744,7 +5752,8 @@ void kvm_mmu_slot_set_dirty(struct kvm *kvm, bool flush; spin_lock(&kvm->mmu_lock); - flush = slot_handle_all_level(kvm, memslot, __rmap_set_dirty, false); + flush = slot_handle_all_level(kvm, memslot, __rmap_set_dirty, false, + NULL); spin_unlock(&kvm->mmu_lock); lockdep_assert_held(&kvm->slots_lock); From patchwork Sun Nov 4 17:11:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ahmed Soliman X-Patchwork-Id: 10666941 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C8B0D15E9 for ; Sun, 4 Nov 2018 17:12:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AFEB8295AE for ; Sun, 4 Nov 2018 17:12:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A3B91295CF; Sun, 4 Nov 2018 17:12:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id D850F295AE for ; Sun, 4 Nov 2018 17:12:51 +0000 (UTC) Received: (qmail 30107 invoked by uid 550); 4 Nov 2018 17:12:32 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 30029 invoked from network); 4 Nov 2018 17:12:32 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=/PEiWOknyyJFIL6QHknHD9RivhxJlGWUrrDq5I+qUo0=; b=I8alelYdRaS7htEJKfw/TWJp+qqBX6UGfMtfbw69N1p0PDaM4P8FKzDrIt63zBGWgu SZPJbHPOIi2zu0K3TopeSsnMwWSfb0R2rK/hAYmR0JEfILZJM4XVfRCT1sU5Vccdv+DY a3SfwWeeBSC61Zw7mHIvnPjnS4MQhUYOW0RJIlhcDyDISdQlvy8iU/WowxaG8PnK2roW gm0UwqgOhrxqRbBe9WURQwSkmo3FOCMEIXyaPAeduwUKmXm0yzUepurh8NU3EhMIUTdM nk0urse7K75QagM8e1nd6L7EU0tekX1lpFUzNOshy37y2vyn2uMcdbZLM3Sev77FsalG Lz5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=/PEiWOknyyJFIL6QHknHD9RivhxJlGWUrrDq5I+qUo0=; b=IdoGFI5tqRh+zJPPrcAVdJDoxQ4FinG5bNvfXvx5p+wQ6677z6kD7s7fJvHb3Mv1F+ wyqpVMv9Uh7BwKqowMph+Q0nw+9mPwGMGe8vK4DOertmoYmh0veZmDhkfq3V3K+wcNkm kWnBDWaIVq7XbFwhgCXC7RE6KgXDIBYTreP2nDhQXXnV8LZMYW5uugr17FX9jkM+Qz21 d4igBl4qHBAaJ24pVHn7J8+DkKniYupRcGEWrd6mX1wuw2ylA38l0z6lbJOGHB+5vUEp javnGlXJqLcW7isbqRodh67caR93cYsFdKFGCMPwyequmTojrNO/LLeu1B8TaRJ1gIZ6 S8jQ== X-Gm-Message-State: AGRZ1gJWga6bWK9Lw17thc/imIsSQIe15cFtgMZu6NbpZCJ/wZ6wYT0S wA7LulpKyT/96k6VySLBoc0= X-Google-Smtp-Source: AJdET5dQiFY9UiMhXkmb90HhcNiWUlbNLxCU3J5X4n0z+Gdgw6s8b4RU073xvVFVBn7SiUe6vVkvSA== X-Received: by 2002:a1c:4407:: with SMTP id r7-v6mr3774993wma.16.1541351540687; Sun, 04 Nov 2018 09:12:20 -0800 (PST) From: Ahmed Abd El Mawgood To: Paolo Bonzini , rkrcmar@redhat.com, Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , hpa@zytor.com, x86@kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, ahmedsoliman0x666@gmail.com, ovich00@gmail.com, kernel-hardening@lists.openwall.com, nigel.edwards@hpe.com, Boris Lukashev , Hossam Hassan <7ossam9063@gmail.com>, "Ahmed Lotfy igor . stoppa @ gmail . com" Subject: [PATCH V6 3/8] KVM: Document Memory ROE Date: Sun, 4 Nov 2018 19:11:19 +0200 Message-Id: <20181104171124.5717-4-ahmedsoliman0x666@gmail.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> References: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> X-Virus-Scanned: ClamAV using ClamSMTP ROE version documented here is implemented in the next 2 patches Signed-off-by: Ahmed Abd El Mawgood --- Documentation/virtual/kvm/hypercalls.txt | 31 ++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/Documentation/virtual/kvm/hypercalls.txt b/Documentation/virtual/kvm/hypercalls.txt index da24c138c8d1..8af64d826f03 100644 --- a/Documentation/virtual/kvm/hypercalls.txt +++ b/Documentation/virtual/kvm/hypercalls.txt @@ -141,3 +141,34 @@ a0 corresponds to the APIC ID in the third argument (a2), bit 1 corresponds to the APIC ID a2+1, and so on. Returns the number of CPUs to which the IPIs were delivered successfully. + +7. KVM_HC_ROE +---------------- +Architecture: x86 +Status: active +Purpose: Hypercall used to apply Read-Only Enforcement to guest memory and +registers +Usage 1: + a0: ROE_VERSION + +Returns non-signed number that represents the current version of ROE +implementation current version. + +Usage 2: + + a0: ROE_MPROTECT (requires version >= 1) + a1: Start address aligned to page boundary. + a2: Number of pages to be protected. + +This configuration lets a guest kernel have part of its read/write memory +converted into read-only. This action is irreversible. +Upon successful run, the number of pages protected is returned. + +Error codes: + -KVM_ENOSYS: system call being triggered from ring 3 or it is not + implemented. + -EINVAL: error based on given parameters. + +Notes: KVM_HC_ROE can not be triggered from guest Ring 3 (user mode). The +reason is that user mode malicious software can make use of it to enforce read +only protection on an arbitrary memory page thus crashing the kernel. From patchwork Sun Nov 4 17:11:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ahmed Soliman X-Patchwork-Id: 10666947 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A7B8813A4 for ; Sun, 4 Nov 2018 17:13:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DC88295AE for ; Sun, 4 Nov 2018 17:13:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8175F295CB; Sun, 4 Nov 2018 17:13:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id 34F0B295AE for ; Sun, 4 Nov 2018 17:13:01 +0000 (UTC) Received: (qmail 30510 invoked by uid 550); 4 Nov 2018 17:12:36 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 30448 invoked from network); 4 Nov 2018 17:12:36 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=5FjwmPvDwP8KWZiMJvZtWu/x/3jKbDOs7MA9bKIwUKs=; b=VNcMEaql14pN5YgCZJKZGeFNOHRrGbucZOZEy/EYB1DPaQkgSPw+eYp/P0LERU/ML/ A9rJjcVeYOyfyir61zbepvVj4w+9KlmxVwiWtcCvKdBRj4ECRCy58G5ULT6Jr3riFwtG HuKnOzriObr2lhXNtXJQw6KEa7X9/EAZdYgLlvRaMln27cZAMxqaiVmhSz+NXBlMpZu0 BKO3h59sXj91K6Bss22b68fxbEPxAyF06KSiGIViqoUrFc3iO6EnmHH80Jbklpq+uHmq zE/SbofEN+kbf5tn47OGlLypOr3vuWy+Valp2MhWhs+geicACC5yatwXZbuGTCGJYgrq 8NvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=5FjwmPvDwP8KWZiMJvZtWu/x/3jKbDOs7MA9bKIwUKs=; b=jc0wflzAc/uomvoOI2vqbCww8Rr+tKo+uz/AhHxRyKJwf1HmkRW7Qia/SlMCTlbV1A BqunmA9A4DdUi06hMKGCzGzKdwlMzp71V4ZrduCm+Gymi+gZa96ZgYigbbr65lQDLIE0 o7lzwhnmwblV/+MmqWEfhUhC7lYDGRYFY+0kUy7b5GwZd7t05n7xIso/VNdidhofkkZq 6wk84kZt7MY0QLcJAdh2Y4BdC8/LXDzE7lzev6c2XfgeMoByg3MOAkU/ElhWWzQUlNlf pAeW4Q0mmbJeZkIQzpGuErmBqlHXwZt6i2l0tIL/k45DIFjsi16JjbVzNg3qNaE0HIxL pcgA== X-Gm-Message-State: AGRZ1gJvmoH2eesoWI23Ap1tYHzE7I2kwgQmNuvHpz3lliz5DbJ9T0jd jJ8o7WHMG6+VrArvlrAPpVk= X-Google-Smtp-Source: AJdET5dNsspXV+pzpVzwUi78opQFqql5RPDtqULpnupyKAMOlHIAKXs5Lj0qY9WdWjH7o0/5Sfu1qg== X-Received: by 2002:adf:f181:: with SMTP id h1-v6mr11614491wro.79.1541351544034; Sun, 04 Nov 2018 09:12:24 -0800 (PST) From: Ahmed Abd El Mawgood To: Paolo Bonzini , rkrcmar@redhat.com, Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , hpa@zytor.com, x86@kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, ahmedsoliman0x666@gmail.com, ovich00@gmail.com, kernel-hardening@lists.openwall.com, nigel.edwards@hpe.com, Boris Lukashev , Hossam Hassan <7ossam9063@gmail.com>, "Ahmed Lotfy igor . stoppa @ gmail . com" Subject: [PATCH V6 4/8] KVM: Create architecture independent ROE skeleton Date: Sun, 4 Nov 2018 19:11:20 +0200 Message-Id: <20181104171124.5717-5-ahmedsoliman0x666@gmail.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> References: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> X-Virus-Scanned: ClamAV using ClamSMTP This patch introduces a hypercall that can assist against subset of kernel rootkits, it works by place readonly protection in shadow PTE. The end result protection is also kept in a bitmap for each kvm_memory_slot and is used as reference when updating SPTEs. The whole goal is to protect the guest kernel static data from modification if attacker is running from guest ring 0, for this reason there is no hypercall to revert effect of Memory ROE hypercall. For this patch to work on a given arch/ one would need to implement 2 function that are architecture specific: kvm_roe_arch_commit_protection() and kvm_roe_arch_is_userspace(). Also it would need to have kvm_roe invoked using the appropriate hypercall mechanism. Signed-off-by: Ahmed Abd El Mawgood --- include/kvm/roe.h | 23 ++++++ include/linux/kvm_host.h | 3 + include/uapi/linux/kvm_para.h | 4 + virt/kvm/kvm_main.c | 19 +++-- virt/kvm/roe.c | 136 ++++++++++++++++++++++++++++++++++ virt/kvm/roe_generic.h | 29 ++++++++ 6 files changed, 209 insertions(+), 5 deletions(-) create mode 100644 include/kvm/roe.h create mode 100644 virt/kvm/roe.c create mode 100644 virt/kvm/roe_generic.h diff --git a/include/kvm/roe.h b/include/kvm/roe.h new file mode 100644 index 000000000000..d93855f509a2 --- /dev/null +++ b/include/kvm/roe.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __KVM_ROE_H__ +#define __KVM_ROE_H__ +/* + * KVM Read Only Enforcement + * Copyright (c) 2018 Ahmed Mohamed Abd El Mawgood + * + * Author Ahmed Mohamed Abd El Mawgood + * + */ +#ifdef CONFIG_KVM_ROE +void kvm_roe_arch_commit_protection(struct kvm *kvm, + struct kvm_memory_slot *slot); +int kvm_roe(struct kvm_vcpu *vcpu, u64 a0, u64 a1, u64 a2, u64 a3); +bool kvm_roe_arch_is_userspace(struct kvm_vcpu *vcpu); +#else +static inline int kvm_roe(struct kvm_vcpu *vcpu, u64 a0, u64 a1, u64 a2, u64 a3) +{ + return -KVM_ENOSYS; +} +#endif +#endif diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index c926698040e0..be6885bc28bc 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -297,6 +297,9 @@ static inline int kvm_vcpu_exiting_guest_mode(struct kvm_vcpu *vcpu) struct kvm_memory_slot { gfn_t base_gfn; unsigned long npages; +#ifdef CONFIG_KVM_ROE + unsigned long *roe_bitmap; +#endif unsigned long *dirty_bitmap; struct kvm_arch_memory_slot arch; unsigned long userspace_addr; diff --git a/include/uapi/linux/kvm_para.h b/include/uapi/linux/kvm_para.h index 6c0ce49931e5..e6004e0750fd 100644 --- a/include/uapi/linux/kvm_para.h +++ b/include/uapi/linux/kvm_para.h @@ -28,7 +28,11 @@ #define KVM_HC_MIPS_CONSOLE_OUTPUT 8 #define KVM_HC_CLOCK_PAIRING 9 #define KVM_HC_SEND_IPI 10 +#define KVM_HC_ROE 11 +/* ROE Functionality parameters */ +#define ROE_VERSION 0 +#define ROE_MPROTECT 1 /* * hypercalls use architecture specific */ diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 039c1ef9a786..814ee0fd3578 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -61,6 +61,7 @@ #include "coalesced_mmio.h" #include "async_pf.h" #include "vfio.h" +#include "roe_generic.h" #define CREATE_TRACE_POINTS #include @@ -552,9 +553,10 @@ static void kvm_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free, struct kvm_memory_slot *dont, enum kvm_mr_change change) { - if (change == KVM_MR_DELETE) + if (change == KVM_MR_DELETE) { + kvm_roe_free(free); kvm_destroy_dirty_bitmap(free); - + } kvm_arch_free_memslot(kvm, free, dont); free->npages = 0; @@ -1020,6 +1022,8 @@ int __kvm_set_memory_region(struct kvm *kvm, if (kvm_create_dirty_bitmap(&new) < 0) goto out_free; } + if (kvm_roe_init(&new) < 0) + goto out_free; slots = kvzalloc(sizeof(struct kvm_memslots), GFP_KERNEL); if (!slots) @@ -1273,13 +1277,18 @@ static bool memslot_is_readonly(struct kvm_memory_slot *slot) return slot->flags & KVM_MEM_READONLY; } +static bool gfn_is_readonly(struct kvm_memory_slot *slot, gfn_t gfn) +{ + return gfn_is_full_roe(slot, gfn) || memslot_is_readonly(slot); +} + static unsigned long __gfn_to_hva_many(struct kvm_memory_slot *slot, gfn_t gfn, gfn_t *nr_pages, bool write) { if (!slot || slot->flags & KVM_MEMSLOT_INVALID) return KVM_HVA_ERR_BAD; - if (memslot_is_readonly(slot) && write) + if (gfn_is_readonly(slot, gfn) && write) return KVM_HVA_ERR_RO_BAD; if (nr_pages) @@ -1327,7 +1336,7 @@ unsigned long gfn_to_hva_memslot_prot(struct kvm_memory_slot *slot, unsigned long hva = __gfn_to_hva_many(slot, gfn, NULL, false); if (!kvm_is_error_hva(hva) && writable) - *writable = !memslot_is_readonly(slot); + *writable = !gfn_is_readonly(slot, gfn); return hva; } @@ -1565,7 +1574,7 @@ kvm_pfn_t __gfn_to_pfn_memslot(struct kvm_memory_slot *slot, gfn_t gfn, } /* Do not map writable pfn in the readonly memslot. */ - if (writable && memslot_is_readonly(slot)) { + if (writable && gfn_is_readonly(slot, gfn)) { *writable = false; writable = NULL; } diff --git a/virt/kvm/roe.c b/virt/kvm/roe.c new file mode 100644 index 000000000000..1a0e4be0198c --- /dev/null +++ b/virt/kvm/roe.c @@ -0,0 +1,136 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * KVM Read Only Enforcement + * Copyright (c) 2018 Ahmed Mohamed Abd El Mawgood + * + * Author: Ahmed Mohamed Abd El Mawgood + * + */ +#include +#include +#include +#include + +int kvm_roe_init(struct kvm_memory_slot *slot) +{ + slot->roe_bitmap = kvzalloc(BITS_TO_LONGS(slot->npages) * + sizeof(unsigned long), GFP_KERNEL); + if (!slot->roe_bitmap) + return -ENOMEM; + return 0; + +} + +void kvm_roe_free(struct kvm_memory_slot *slot) +{ + kvfree(slot->roe_bitmap); +} + +static void kvm_roe_protect_slot(struct kvm *kvm, struct kvm_memory_slot *slot, + gfn_t gfn, u64 npages) +{ + int i; + + for (i = gfn - slot->base_gfn; i < gfn + npages - slot->base_gfn; i++) + set_bit(i, slot->roe_bitmap); + kvm_roe_arch_commit_protection(kvm, slot); +} + + +static int __kvm_roe_protect_range(struct kvm *kvm, gpa_t gpa, u64 npages) +{ + struct kvm_memory_slot *slot; + gfn_t gfn = gpa >> PAGE_SHIFT; + int count = 0; + + while (npages != 0) { + slot = gfn_to_memslot(kvm, gfn); + if (!slot) { + gfn += 1; + npages -= 1; + continue; + } + if (gfn + npages > slot->base_gfn + slot->npages) { + u64 _npages = slot->base_gfn + slot->npages - gfn; + + kvm_roe_protect_slot(kvm, slot, gfn, _npages); + gfn += _npages; + count += _npages; + npages -= _npages; + } else { + kvm_roe_protect_slot(kvm, slot, gfn, npages); + count += npages; + npages = 0; + } + } + if (count == 0) + return -EINVAL; + return count; +} + +static int kvm_roe_protect_range(struct kvm *kvm, gpa_t gpa, u64 npages) +{ + int r; + + mutex_lock(&kvm->slots_lock); + r = __kvm_roe_protect_range(kvm, gpa, npages); + mutex_unlock(&kvm->slots_lock); + return r; +} + + +static int kvm_roe_full_protect_range(struct kvm_vcpu *vcpu, u64 gva, + u64 npages) +{ + struct kvm *kvm = vcpu->kvm; + gpa_t gpa; + u64 hva; + u64 count = 0; + int i; + int status; + + if (gva & ~PAGE_MASK) + return -EINVAL; + // We need to make sure that there will be no overflow + if ((npages << PAGE_SHIFT) >> PAGE_SHIFT != npages || npages == 0) + return -EINVAL; + for (i = 0; i < npages; i++) { + gpa = kvm_mmu_gva_to_gpa_system(vcpu, gva + (i << PAGE_SHIFT), + NULL); + hva = gfn_to_hva(kvm, gpa >> PAGE_SHIFT); + if (kvm_is_error_hva(hva)) + continue; + if (!access_ok(VERIFY_WRITE, hva, 1 << PAGE_SHIFT)) + continue; + status = kvm_roe_protect_range(vcpu->kvm, gpa, 1); + if (status > 0) + count += status; + } + if (count == 0) + return -EINVAL; + return count; +} + +int kvm_roe(struct kvm_vcpu *vcpu, u64 a0, u64 a1, u64 a2, u64 a3) +{ + int ret; + /* + * First we need to make sure that we are running from something that + * isn't usermode + */ + if (kvm_roe_arch_is_userspace(vcpu)) + return -KVM_ENOSYS; + switch (a0) { + case ROE_VERSION: + ret = 1; //current version + break; + case ROE_MPROTECT: + ret = kvm_roe_full_protect_range(vcpu, a1, a2); + break; + default: + ret = -EINVAL; + } + return ret; +} +EXPORT_SYMBOL_GPL(kvm_roe); diff --git a/virt/kvm/roe_generic.h b/virt/kvm/roe_generic.h new file mode 100644 index 000000000000..42d6ad3a4971 --- /dev/null +++ b/virt/kvm/roe_generic.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __KVM_ROE_GENERIC_H__ +#define __KVM_ROE_GENERIC_H__ +/* + * KVM Read Only Enforcement + * Copyright (c) 2018 Ahmed Mohamed Abd El Mawgood + * + * Author Ahmed Mohamed Abd El Mawgood + * + */ +#ifdef CONFIG_KVM_ROE + +void kvm_roe_free(struct kvm_memory_slot *slot); +int kvm_roe_init(struct kvm_memory_slot *slot); +static inline bool gfn_is_full_roe(struct kvm_memory_slot *slot, gfn_t gfn) +{ + return test_bit(gfn - slot->base_gfn, slot->roe_bitmap); +} +#else +static void kvm_roe_free(struct kvm_memory_slot *slot) {} +static int kvm_roe_init(struct kvm_memory_slot *slot) { return 0; } +static inline bool gfn_is_full_roe(struct kvm_memory_slot *slot, gfn_t gfn) +{ + return false; +} +#endif + +#endif From patchwork Sun Nov 4 17:11:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ahmed Soliman X-Patchwork-Id: 10666955 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D04B913A4 for ; Sun, 4 Nov 2018 17:13:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B6D51295AE for ; Sun, 4 Nov 2018 17:13:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AABD7295CB; Sun, 4 Nov 2018 17:13:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id BD6E0295AE for ; Sun, 4 Nov 2018 17:13:14 +0000 (UTC) Received: (qmail 31858 invoked by uid 550); 4 Nov 2018 17:12:39 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 31788 invoked from network); 4 Nov 2018 17:12:38 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=seyTtoSTwgkRB9A1VPkACGfqai/cXOBP4zh5/ojOMcE=; b=T0/JMBvomR+UgEUQXk9ppNZwzQePol5YoyOWfXYx+E1pbzUpVdKTCUTkIA6ws0rpua zK477ObgAQ2bXZXKbCxFJUsSY1U+mKjUXvaf/mSiWVoZg+/M+f6DTmnWV6tZzckAbmOd l8fhYdzjB47hLh7p0JB6MDGynj8bNWdggsYeLTi0qEXbp38YDDsomviHqaK0H5nAfYOf QCnWqyIryNGdxCCIunnFKGcXry//xT6Un1Lkxvsp7KWEGPDgNK8cp1GzzUGYx87/Mfu6 8cIW6vLypS23XkqxV0yGhs3qbV/cJYZZ8S/XzELIvmuPCvELea0ICZIwXo3Y1N5o/O5f faiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=seyTtoSTwgkRB9A1VPkACGfqai/cXOBP4zh5/ojOMcE=; b=QvfB2wncJAikq3crsEbjwn75GKtPGY3dtvqKqMWzli8S5u7AoIWXBH0G6SR8AJ1VHf 5upH+U1v10LNIvCpXCcooCcqqlzTqyEn7ryS6Gah96SnseGQt8A781SxEL3VWdKInVlG yCVe+eS13xu6d3jmme4cVZqJ8u6qhtJddJ1zF09IMu7TxVuB/MnGZwMf2yo6qSYckfV3 exWYyu2F4nV4RlWOLQEqHTbxzPURZXldJHTf/4k/lET4Ib4NICbaTPX8u0KL/qkZOFCM MtybnVWKGToL3LZS3gSbnu3s4klq0V3nwS7VnuoAABMGfCGdskovwYDhUoQwT5xDrJHW /MBQ== X-Gm-Message-State: AGRZ1gLrww3lQiFVN6AU8bS20kDZNgydLdDgwiLJzwBht8FI0bW3iyjd 8A0V4/mT9k0vWn09a3E8Poo= X-Google-Smtp-Source: AJdET5d6B5o7n2MID4KJz+pYPqXqsDUBIhwOTSkVOu61PXuILWGC49C5RqNwgDYAZL7VisIg0g4Ftg== X-Received: by 2002:a1c:410b:: with SMTP id o11-v6mr3878868wma.49.1541351547166; Sun, 04 Nov 2018 09:12:27 -0800 (PST) From: Ahmed Abd El Mawgood To: Paolo Bonzini , rkrcmar@redhat.com, Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , hpa@zytor.com, x86@kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, ahmedsoliman0x666@gmail.com, ovich00@gmail.com, kernel-hardening@lists.openwall.com, nigel.edwards@hpe.com, Boris Lukashev , Hossam Hassan <7ossam9063@gmail.com>, "Ahmed Lotfy igor . stoppa @ gmail . com" Subject: [PATCH V6 5/8] KVM: X86: Enable ROE for x86 Date: Sun, 4 Nov 2018 19:11:21 +0200 Message-Id: <20181104171124.5717-6-ahmedsoliman0x666@gmail.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> References: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> X-Virus-Scanned: ClamAV using ClamSMTP This patch implements kvm_roe_arch_commit_protection and kvm_roe_arch_is_userspace for x86, and invoke kvm_roe via the appropriate vmcall. Signed-off-by: Ahmed Abd El Mawgood --- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/Kconfig | 8 +++ arch/x86/kvm/Makefile | 4 +- arch/x86/kvm/mmu.c | 61 ++++++++---------- arch/x86/kvm/mmu.h | 40 +++++++++++- arch/x86/kvm/roe.c | 106 ++++++++++++++++++++++++++++++++ arch/x86/kvm/roe_arch.h | 50 +++++++++++++++ arch/x86/kvm/x86.c | 11 ++-- 8 files changed, 237 insertions(+), 45 deletions(-) create mode 100644 arch/x86/kvm/roe.c create mode 100644 arch/x86/kvm/roe_arch.h diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 55e51ff7e421..eefa2e8c7c44 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1229,7 +1229,7 @@ void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask, u64 acc_track_mask, u64 me_mask); void kvm_mmu_reset_context(struct kvm_vcpu *vcpu); -void kvm_mmu_slot_remove_write_access(struct kvm *kvm, +void kvm_mmu_slot_apply_write_access(struct kvm *kvm, struct kvm_memory_slot *memslot); void kvm_mmu_zap_collapsible_sptes(struct kvm *kvm, const struct kvm_memory_slot *memslot); diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index 1bbec387d289..390a2481efdd 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -96,6 +96,14 @@ config KVM_MMU_AUDIT This option adds a R/W kVM module parameter 'mmu_audit', which allows auditing of KVM MMU events at runtime. +config KVM_ROE + def_bool y + bool "Hypercall Memory Read-Only Enforcement" + depends on KVM && X86 + help + This option adds KVM_HC_ROE hypercall to kvm as a hardening + mechanism to protect memory pages from being edited. + # OK, it's a little counter-intuitive to do this, but it puts it neatly under # the virtualization menu. source drivers/vhost/Kconfig diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile index dc4f2fdf5e57..8b359bc51b3e 100644 --- a/arch/x86/kvm/Makefile +++ b/arch/x86/kvm/Makefile @@ -9,8 +9,10 @@ CFLAGS_vmx.o := -I. KVM := ../../../virt/kvm kvm-y += $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o \ - $(KVM)/eventfd.o $(KVM)/irqchip.o $(KVM)/vfio.o + $(KVM)/eventfd.o $(KVM)/irqchip.o $(KVM)/vfio.o + kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o +kvm-$(CONFIG_KVM_ROE) += $(KVM)/roe.o roe.o kvm-y += x86.o mmu.o emulate.o i8259.o irq.o lapic.o \ i8254.o ioapic.o irq_comm.o cpuid.o pmu.o mtrr.o \ diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index c54ec914935b..4e6887ddfe31 100644 --- a/arch/x86/kvm/mmu.c +++ b/arch/x86/kvm/mmu.c @@ -23,7 +23,7 @@ #include "x86.h" #include "kvm_cache_regs.h" #include "cpuid.h" - +#include "roe_arch.h" #include #include #include @@ -1307,8 +1307,8 @@ static void pte_list_remove(struct kvm_rmap_head *rmap_head, u64 *sptep) __pte_list_remove(sptep, rmap_head); } -static struct kvm_rmap_head *__gfn_to_rmap(gfn_t gfn, int level, - struct kvm_memory_slot *slot) +struct kvm_rmap_head *__gfn_to_rmap(gfn_t gfn, int level, + struct kvm_memory_slot *slot) { unsigned long idx; @@ -1358,16 +1358,6 @@ static void rmap_remove(struct kvm *kvm, u64 *spte) __pte_list_remove(spte, rmap_head); } -/* - * Used by the following functions to iterate through the sptes linked by a - * rmap. All fields are private and not assumed to be used outside. - */ -struct rmap_iterator { - /* private fields */ - struct pte_list_desc *desc; /* holds the sptep if not NULL */ - int pos; /* index of the sptep */ -}; - /* * Iteration must be started by this function. This should also be used after * removing/dropping sptes from the rmap link because in such cases the @@ -1375,8 +1365,7 @@ struct rmap_iterator { * * Returns sptep if found, NULL otherwise. */ -static u64 *rmap_get_first(struct kvm_rmap_head *rmap_head, - struct rmap_iterator *iter) +u64 *rmap_get_first(struct kvm_rmap_head *rmap_head, struct rmap_iterator *iter) { u64 *sptep; @@ -1402,7 +1391,7 @@ static u64 *rmap_get_first(struct kvm_rmap_head *rmap_head, * * Returns sptep if found, NULL otherwise. */ -static u64 *rmap_get_next(struct rmap_iterator *iter) +u64 *rmap_get_next(struct rmap_iterator *iter) { u64 *sptep; @@ -1473,7 +1462,7 @@ static void drop_large_spte(struct kvm_vcpu *vcpu, u64 *sptep) * * Return true if tlb need be flushed. */ -static bool spte_write_protect(u64 *sptep, bool pt_protect) +bool spte_write_protect(u64 *sptep, bool pt_protect) { u64 spte = *sptep; @@ -1491,8 +1480,7 @@ static bool spte_write_protect(u64 *sptep, bool pt_protect) } static bool __rmap_write_protect(struct kvm *kvm, - struct kvm_rmap_head *rmap_head, - bool pt_protect, void *data) + struct kvm_rmap_head *rmap_head, bool pt_protect) { u64 *sptep; struct rmap_iterator iter; @@ -1591,7 +1579,7 @@ static void kvm_mmu_write_protect_pt_masked(struct kvm *kvm, while (mask) { rmap_head = __gfn_to_rmap(slot->base_gfn + gfn_offset + __ffs(mask), PT_PAGE_TABLE_LEVEL, slot); - __rmap_write_protect(kvm, rmap_head, false, NULL); + __rmap_write_protect(kvm, rmap_head, false); /* clear the first set bit */ mask &= mask - 1; @@ -1661,17 +1649,17 @@ int kvm_arch_write_log_dirty(struct kvm_vcpu *vcpu) return 0; } -bool kvm_mmu_slot_gfn_write_protect(struct kvm *kvm, + +bool kvm_mmu_slot_gfn_write_protect_old(struct kvm *kvm, struct kvm_memory_slot *slot, u64 gfn) { struct kvm_rmap_head *rmap_head; int i; bool write_protected = false; - for (i = PT_PAGE_TABLE_LEVEL; i <= PT_MAX_HUGEPAGE_LEVEL; ++i) { rmap_head = __gfn_to_rmap(gfn, i, slot); - write_protected |= __rmap_write_protect(kvm, rmap_head, true, - NULL); + write_protected |= __rmap_write_protect(kvm, rmap_head, + true); } return write_protected; @@ -5526,10 +5514,6 @@ void kvm_mmu_uninit_vm(struct kvm *kvm) kvm_page_track_unregister_notifier(kvm, node); } -/* The return value indicates if tlb flush on all vcpus is needed. */ -typedef bool (*slot_level_handler) (struct kvm *kvm, - struct kvm_rmap_head *rmap_head, void *data); - /* The caller should hold mmu-lock before calling this function. */ static __always_inline bool slot_handle_level_range(struct kvm *kvm, struct kvm_memory_slot *memslot, @@ -5573,9 +5557,8 @@ slot_handle_level(struct kvm *kvm, struct kvm_memory_slot *memslot, lock_flush_tlb, data); } -static __always_inline bool -slot_handle_all_level(struct kvm *kvm, struct kvm_memory_slot *memslot, - slot_level_handler fn, bool lock_flush_tlb, void *data) +bool slot_handle_all_level(struct kvm *kvm, struct kvm_memory_slot *memslot, + slot_level_handler fn, bool lock_flush_tlb, void *data) { return slot_handle_level(kvm, memslot, fn, PT_PAGE_TABLE_LEVEL, PT_MAX_HUGEPAGE_LEVEL, lock_flush_tlb, data); @@ -5627,11 +5610,10 @@ static bool slot_rmap_write_protect(struct kvm *kvm, struct kvm_rmap_head *rmap_head, void *data) { - return __rmap_write_protect(kvm, rmap_head, false, data); + return __rmap_write_protect(kvm, rmap_head, false); } -void kvm_mmu_slot_remove_write_access(struct kvm *kvm, - struct kvm_memory_slot *memslot) +bool protect_all_levels_old(struct kvm *kvm, struct kvm_memory_slot *memslot) { bool flush; @@ -5639,9 +5621,14 @@ void kvm_mmu_slot_remove_write_access(struct kvm *kvm, flush = slot_handle_all_level(kvm, memslot, slot_rmap_write_protect, false, NULL); spin_unlock(&kvm->mmu_lock); - + return flush; +} +void kvm_mmu_slot_apply_write_access(struct kvm *kvm, + struct kvm_memory_slot *memslot) +{ + bool flush = protect_all_levels(kvm, memslot); /* - * kvm_mmu_slot_remove_write_access() and kvm_vm_ioctl_get_dirty_log() + * kvm_mmu_slot_apply_write_access() and kvm_vm_ioctl_get_dirty_log() * which do tlb flush out of mmu-lock should be serialized by * kvm->slots_lock otherwise tlb flush would be missed. */ @@ -5738,7 +5725,7 @@ void kvm_mmu_slot_largepage_remove_write_access(struct kvm *kvm, false, NULL); spin_unlock(&kvm->mmu_lock); - /* see kvm_mmu_slot_remove_write_access */ + /* see kvm_mmu_slot_apply_write_access*/ lockdep_assert_held(&kvm->slots_lock); if (flush) diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index c7b333147c4a..23cf58062546 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -4,7 +4,6 @@ #include #include "kvm_cache_regs.h" - #define PT64_PT_BITS 9 #define PT64_ENT_PER_PAGE (1 << PT64_PT_BITS) #define PT32_PT_BITS 10 @@ -43,6 +42,24 @@ #define PT32_ROOT_LEVEL 2 #define PT32E_ROOT_LEVEL 3 +#define for_each_rmap_spte(_rmap_head_, _iter_, _spte_) \ + for (_spte_ = rmap_get_first(_rmap_head_, _iter_); \ + _spte_; _spte_ = rmap_get_next(_iter_)) + +/* + * Used by the following functions to iterate through the sptes linked by a + * rmap. All fields are private and not assumed to be used outside. + */ +struct rmap_iterator { + /* private fields */ + struct pte_list_desc *desc; /* holds the sptep if not NULL */ + int pos; /* index of the sptep */ +}; + +u64 *rmap_get_first(struct kvm_rmap_head *rmap_head, + struct rmap_iterator *iter); +u64 *rmap_get_next(struct rmap_iterator *iter); +bool spte_write_protect(u64 *sptep, bool pt_protect); static inline u64 rsvd_bits(int s, int e) { if (e < s) @@ -203,12 +220,31 @@ static inline u8 permission_fault(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, return -(u32)fault & errcode; } +/* The return value indicates if tlb flush on all vcpus is needed. */ +typedef bool (*slot_level_handler) (struct kvm *kvm, + struct kvm_rmap_head *rmap_head, void *data); + void kvm_mmu_invalidate_zap_all_pages(struct kvm *kvm); void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end); void kvm_mmu_gfn_disallow_lpage(struct kvm_memory_slot *slot, gfn_t gfn); void kvm_mmu_gfn_allow_lpage(struct kvm_memory_slot *slot, gfn_t gfn); -bool kvm_mmu_slot_gfn_write_protect(struct kvm *kvm, +bool kvm_mmu_slot_gfn_write_protect_old(struct kvm *kvm, struct kvm_memory_slot *slot, u64 gfn); int kvm_arch_write_log_dirty(struct kvm_vcpu *vcpu); +bool protect_all_levels_old(struct kvm *kvm, struct kvm_memory_slot *memslot); +bool slot_handle_all_level(struct kvm *kvm, struct kvm_memory_slot *memslot, + slot_level_handler fn, bool lock_flush_tlb, void *data); +struct kvm_rmap_head *__gfn_to_rmap(gfn_t gfn, int level, + struct kvm_memory_slot *slot); +/* + * This include line **must** be the last line in this file, here is why + * some functions have 2 versions fcn_old() vs fcn_roe() the old functions is + * old in the sence of it was already there. Now to resolve the issue of + * #ifdef CONFIG_KVM_ROE everywhere there is static inline functions that + * resolve fcn() into either fcn_old or fcn_roe() that are placed in roe_arch.h + * I had 2 options first is move all those functions with there #ifdef to here + * or include "roe_arch.h". I chose the later one + */ +#include "roe_arch.h" #endif diff --git a/arch/x86/kvm/roe.c b/arch/x86/kvm/roe.c new file mode 100644 index 000000000000..cd3e6944c15f --- /dev/null +++ b/arch/x86/kvm/roe.c @@ -0,0 +1,106 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * KVM Read Only Enforcement + * Copyright (c) 2018 Ahmed Mohamed Abd El Mawgood + * + * Author Ahmed Mohamed Abd El Mawgood + * + */ +#include +#include +#include + + +#include +#include "kvm_cache_regs.h" +#include "mmu.h" +#include "roe_arch.h" + +static bool __rmap_write_protect_roe(struct kvm *kvm, + struct kvm_rmap_head *rmap_head, bool pt_protect, + struct kvm_write_access_data *d) +{ + u64 *sptep; + struct rmap_iterator iter; + bool prot; + bool flush = false; + + for_each_rmap_spte(rmap_head, &iter, sptep) { + prot = !test_bit(d->i, d->memslot->roe_bitmap) && pt_protect; + flush |= spte_write_protect(sptep, prot); + d->i++; + } + return flush; +} + +bool kvm_mmu_slot_gfn_write_protect_roe(struct kvm *kvm, + struct kvm_memory_slot *slot, u64 gfn) +{ + struct kvm_rmap_head *rmap_head; + int i; + bool write_protected = false; + struct kvm_write_access_data data = { + .i = 0, + .memslot = slot, + }; + for (i = PT_PAGE_TABLE_LEVEL; i <= PT_MAX_HUGEPAGE_LEVEL; ++i) { + rmap_head = __gfn_to_rmap(gfn, i, slot); + write_protected |= __rmap_write_protect_roe(kvm, rmap_head, + true, &data); + } + return write_protected; +} + +static bool slot_rmap_apply_protection(struct kvm *kvm, + struct kvm_rmap_head *rmap_head, void *data) +{ + struct kvm_write_access_data *d = (struct kvm_write_access_data *) data; + bool prot_mask = !(d->memslot->flags & KVM_MEM_READONLY); + + return __rmap_write_protect_roe(kvm, rmap_head, prot_mask, d); +} + +bool roe_protect_all_levels(struct kvm *kvm, struct kvm_memory_slot *memslot) +{ + bool flush; + struct kvm_write_access_data data = { + .i = 0, + .memslot = memslot, + }; + spin_lock(&kvm->mmu_lock); + flush = slot_handle_all_level(kvm, memslot, slot_rmap_apply_protection, + false, &data); + spin_unlock(&kvm->mmu_lock); + return flush; +} + +void kvm_roe_arch_commit_protection(struct kvm *kvm, + struct kvm_memory_slot *slot) +{ + kvm_mmu_slot_apply_write_access(kvm, slot); + kvm_arch_flush_shadow_memslot(kvm, slot); +} +EXPORT_SYMBOL_GPL(kvm_roe_arch_commit_protection); + +bool kvm_roe_arch_is_userspace(struct kvm_vcpu *vcpu) +{ + u64 rflags; + u64 cr0 = kvm_read_cr0(vcpu); + u64 iopl; + + // first checking we are not in protected mode + if ((cr0 & 1) == 0) + return false; + /* + * we don't need to worry about comments in __get_regs + * because we are sure that this function will only be + * triggered at the end of a hypercall instruction. + */ + rflags = kvm_get_rflags(vcpu); + iopl = (rflags >> 12) & 3; + if (iopl != 3) + return false; + return true; +} +EXPORT_SYMBOL_GPL(kvm_roe_arch_is_userspace); diff --git a/arch/x86/kvm/roe_arch.h b/arch/x86/kvm/roe_arch.h new file mode 100644 index 000000000000..41c496be4344 --- /dev/null +++ b/arch/x86/kvm/roe_arch.h @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __KVM_ROE_HARCH_H__ +#define __KVM_ROE_HARCH_H__ +/* + * KVM Read Only Enforcement + * Copyright (c) 2018 Ahmed Mohamed Abd El Mawgood + * + * Author Ahmed Mohamed Abd El Mawgood + * + */ +#include "mmu.h" +#ifdef CONFIG_KVM_ROE + +/* + * This is internal structure used to be be able to access kvm memory slot and + * have track of the number of current PTE when doing shadow PTE walk + */ +struct kvm_write_access_data { + int i; + struct kvm_memory_slot *memslot; +}; +bool roe_protect_all_levels(struct kvm *kvm, struct kvm_memory_slot *memslot); + +static inline bool protect_all_levels(struct kvm *kvm, + struct kvm_memory_slot *memslot) +{ + return roe_protect_all_levels(kvm, memslot); +} +bool kvm_mmu_slot_gfn_write_protect_roe(struct kvm *kvm, + struct kvm_memory_slot *slot, u64 gfn); +static inline bool kvm_mmu_slot_gfn_write_protect(struct kvm *kvm, + struct kvm_memory_slot *slot, u64 gfn) +{ + return kvm_mmu_slot_gfn_write_protect_roe(kvm, slot, gfn); +} +#else +static inline bool protect_all_levels(struct kvm *kvm, + struct kvm_memory_slot *memslot) +{ + return protect_all_levels_old(kvm, memslot); +} +static inline bool kvm_mmu_slot_gfn_write_protect(struct kvm *kvm, + struct kvm_memory_slot *slot, u64 gfn) +{ + return kvm_mmu_slot_gfn_write_protect_old(kvm, slot, gfn); +} + +#endif +#endif diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 66d66d77caee..8510988ead61 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -20,6 +20,7 @@ */ #include +#include #include "irq.h" #include "mmu.h" #include "i8254.h" @@ -4409,7 +4410,7 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log) /* * All the TLBs can be flushed out of mmu lock, see the comments in - * kvm_mmu_slot_remove_write_access(). + * kvm_mmu_slot_apply_write_access(). */ lockdep_assert_held(&kvm->slots_lock); if (is_dirty) @@ -6927,7 +6928,6 @@ static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr, return ret; } #endif - /* * kvm_pv_kick_cpu_op: Kick a vcpu. * @@ -6999,6 +6999,9 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) ret = kvm_pv_send_ipi(vcpu->kvm, a0, a1, a2, a3, op_64_bit); break; #endif + case KVM_HC_ROE: + ret = kvm_roe(vcpu, a0, a1, a2, a3); + break; default: ret = -KVM_ENOSYS; break; @@ -9261,8 +9264,8 @@ static void kvm_mmu_slot_apply_flags(struct kvm *kvm, struct kvm_memory_slot *new) { /* Still write protect RO slot */ + kvm_mmu_slot_apply_write_access(kvm, new); if (new->flags & KVM_MEM_READONLY) { - kvm_mmu_slot_remove_write_access(kvm, new); return; } @@ -9300,7 +9303,7 @@ static void kvm_mmu_slot_apply_flags(struct kvm *kvm, if (kvm_x86_ops->slot_enable_log_dirty) kvm_x86_ops->slot_enable_log_dirty(kvm, new); else - kvm_mmu_slot_remove_write_access(kvm, new); + kvm_mmu_slot_apply_write_access(kvm, new); } else { if (kvm_x86_ops->slot_disable_log_dirty) kvm_x86_ops->slot_disable_log_dirty(kvm, new); From patchwork Sun Nov 4 17:11:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ahmed Soliman X-Patchwork-Id: 10666959 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 22CE715E9 for ; Sun, 4 Nov 2018 17:13:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 051E3289F4 for ; Sun, 4 Nov 2018 17:13:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E71C228A57; Sun, 4 Nov 2018 17:13:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id 35512289F4 for ; Sun, 4 Nov 2018 17:13:28 +0000 (UTC) Received: (qmail 32102 invoked by uid 550); 4 Nov 2018 17:12:42 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 32043 invoked from network); 4 Nov 2018 17:12:41 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=sR5YS7tQy4Qw75ViaeMp3L8IWcGln7wfkKWaGHuSFWE=; b=lWr3sWKvjDoe6kMeJ4m78W+MlXwBaSDUFscSg/Dvb8EJI1bPmZgsUjsSdhj9EJAbzb AYad6ZUh+jDYyC0cn/fppgT+XuVBabxE+R/YlwGFxY/NqFd/tV0Pz8MvTLEsDZJJ/2YH Rghomo+uFV84haI4CO03nlsHZiPVD9NzjyX+/ryck6JO6Yl8HQfXGbUen5tm8BSkCVlP hhY8OKOslkb2EJ+ufUPBFG1kropyaHiTPf0l/AWEG3pkxk+IcQbex57Yl761nYJgrWbW RaUD4wvJeHMUvRbgpoMe/vSGNjMADcTpqgyZjLybrTadBFnw5R3kMWHiibloR6Cp44wN fy/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=sR5YS7tQy4Qw75ViaeMp3L8IWcGln7wfkKWaGHuSFWE=; b=d/Wh89agM1Dh9FMfVKt8jI7N9CcPqdVwRFLxStQH2W54SO/xWEAELovyNoQwKQVi3g C9X1ka1bW36n7xrjKiMBPMy2XWRGBjtdoAtoCZtoT+w2uViz5ckDRvx4jevh89fY46U5 qmB6l6ETVAbNV+8GlVSsb13dCSVeEsen3i+Vp87osrmLBmnO/HNZLNPlRH9J64dSz3aW LYye3ssiQK4mJ4RDQa05TfzThcHbw9rpGWz7W1GI4brYDrHEA0Fju3Vuh0m5HUo36n+V RTUb9cB9Jx3j+Qw3o2whyRZnA48eXW1pPtfN+OBGoltjNZu/rgbjL5Qhl+VQ4ugtJQuj KS4g== X-Gm-Message-State: AGRZ1gJq8seuQUDLi1Ldlz1cgBgPB9CSnZnnhvoF7WziSeqmnqmEwBIu cX7FiZVS6BLYBz10OSfu02U= X-Google-Smtp-Source: AJdET5ffi2FN/aVTU8/FOaMS/7YVSc1g4lQPwCFz06TTueMqm65307GNr/wWiGePLuQ3Z0l7vEh41g== X-Received: by 2002:adf:dbc3:: with SMTP id e3-v6mr17610461wrj.165.1541351550075; Sun, 04 Nov 2018 09:12:30 -0800 (PST) From: Ahmed Abd El Mawgood To: Paolo Bonzini , rkrcmar@redhat.com, Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , hpa@zytor.com, x86@kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, ahmedsoliman0x666@gmail.com, ovich00@gmail.com, kernel-hardening@lists.openwall.com, nigel.edwards@hpe.com, Boris Lukashev , Hossam Hassan <7ossam9063@gmail.com>, "Ahmed Lotfy igor . stoppa @ gmail . com" Subject: [PATCH V6 6/8] KVM: Add support for byte granular memory ROE Date: Sun, 4 Nov 2018 19:11:22 +0200 Message-Id: <20181104171124.5717-7-ahmedsoliman0x666@gmail.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> References: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> X-Virus-Scanned: ClamAV using ClamSMTP This patch documents and implements ROE_MPROTECT_CHUNK, a part of ROE hypercall designed to protect regions of a memory page with byte granularity. This feature provides a key primitive to protect against attacks involving pages remapping. Signed-off-by: Ahmed Abd El Mawgood --- Documentation/virtual/kvm/hypercalls.txt | 9 + include/linux/kvm_host.h | 26 +++ include/uapi/linux/kvm_para.h | 1 + virt/kvm/kvm_main.c | 22 ++- virt/kvm/roe.c | 212 +++++++++++++++++++++-- virt/kvm/roe_generic.h | 15 ++ 6 files changed, 271 insertions(+), 14 deletions(-) diff --git a/Documentation/virtual/kvm/hypercalls.txt b/Documentation/virtual/kvm/hypercalls.txt index 8af64d826f03..a31f316ce6e6 100644 --- a/Documentation/virtual/kvm/hypercalls.txt +++ b/Documentation/virtual/kvm/hypercalls.txt @@ -164,6 +164,15 @@ This configuration lets a guest kernel have part of its read/write memory converted into read-only. This action is irreversible. Upon successful run, the number of pages protected is returned. +Usage 3: + a0: ROE_MPROTECT_CHUNK (requires version >= 2) + a1: Start address aligned to page boundary. + a2: Number of bytes to be protected. +This configuration lets a guest kernel have part of its read/write memory +converted into read-only with bytes granularity. ROE_MPROTECT_CHUNK is +relatively slow compared to ROE_MPROTECT. This action is irreversible. +Upon successful run, the number of bytes protected is returned. + Error codes: -KVM_ENOSYS: system call being triggered from ring 3 or it is not implemented. diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index be6885bc28bc..a6749a52386b 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -294,11 +294,37 @@ static inline int kvm_vcpu_exiting_guest_mode(struct kvm_vcpu *vcpu) */ #define KVM_MEM_MAX_NR_PAGES ((1UL << 31) - 1) +#ifdef CONFIG_KVM_ROE +/* + * This structure is used to hold memory areas that are to be protected in a + * memory frame with mixed page permissions. + **/ +struct protected_chunk { + gpa_t gpa; + u64 size; + struct list_head list; +}; + +static inline bool kvm_roe_range_overlap(struct protected_chunk *chunk, + gpa_t gpa, int len) { + /* + * https://stackoverflow.com/questions/325933/ + * determine-whether-two-date-ranges-overlap + * Assuming that it works, that link ^ provides a solution that is + * better than anything I would ever come up with. + */ + return (gpa <= chunk->gpa + chunk->size - 1) && + (gpa + len - 1 >= chunk->gpa); +} +#endif + struct kvm_memory_slot { gfn_t base_gfn; unsigned long npages; #ifdef CONFIG_KVM_ROE unsigned long *roe_bitmap; + unsigned long *partial_roe_bitmap; + struct list_head *prot_list; #endif unsigned long *dirty_bitmap; struct kvm_arch_memory_slot arch; diff --git a/include/uapi/linux/kvm_para.h b/include/uapi/linux/kvm_para.h index e6004e0750fd..4a84f974bc58 100644 --- a/include/uapi/linux/kvm_para.h +++ b/include/uapi/linux/kvm_para.h @@ -33,6 +33,7 @@ /* ROE Functionality parameters */ #define ROE_VERSION 0 #define ROE_MPROTECT 1 +#define ROE_MPROTECT_CHUNK 2 /* * hypercalls use architecture specific */ diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 814ee0fd3578..48c5d9d9474e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1279,18 +1279,19 @@ static bool memslot_is_readonly(struct kvm_memory_slot *slot) static bool gfn_is_readonly(struct kvm_memory_slot *slot, gfn_t gfn) { - return gfn_is_full_roe(slot, gfn) || memslot_is_readonly(slot); + return gfn_is_full_roe(slot, gfn) || + gfn_is_partial_roe(slot, gfn) || + memslot_is_readonly(slot); } + static unsigned long __gfn_to_hva_many(struct kvm_memory_slot *slot, gfn_t gfn, gfn_t *nr_pages, bool write) { if (!slot || slot->flags & KVM_MEMSLOT_INVALID) return KVM_HVA_ERR_BAD; - if (gfn_is_readonly(slot, gfn) && write) return KVM_HVA_ERR_RO_BAD; - if (nr_pages) *nr_pages = slot->npages - (gfn - slot->base_gfn); @@ -1852,14 +1853,27 @@ int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu, gpa_t gpa, return __kvm_read_guest_atomic(slot, gfn, data, offset, len); } EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_atomic); +static u64 roe_gfn_to_hva(struct kvm_memory_slot *slot, gfn_t gfn, int offset, + int len) +{ + u64 addr; + if (kvm_roe_check_range(slot, gfn, offset, len)) + return KVM_HVA_ERR_RO_BAD; + if (memslot_is_readonly(slot)) + return KVM_HVA_ERR_RO_BAD; + if (gfn_is_full_roe(slot, gfn)) + return KVM_HVA_ERR_RO_BAD; + addr = __gfn_to_hva_many(slot, gfn, NULL, false); + return addr; +} static int __kvm_write_guest_page(struct kvm_memory_slot *memslot, gfn_t gfn, const void *data, int offset, int len) { int r; unsigned long addr; - addr = gfn_to_hva_memslot(memslot, gfn); + addr = roe_gfn_to_hva(memslot, gfn, offset, len); if (kvm_is_error_hva(addr)) return -EFAULT; r = __copy_to_user((void __user *)addr + offset, data, len); diff --git a/virt/kvm/roe.c b/virt/kvm/roe.c index 1a0e4be0198c..e94314fed3a3 100644 --- a/virt/kvm/roe.c +++ b/virt/kvm/roe.c @@ -11,34 +11,89 @@ #include #include #include +#include "roe_generic.h" int kvm_roe_init(struct kvm_memory_slot *slot) { slot->roe_bitmap = kvzalloc(BITS_TO_LONGS(slot->npages) * sizeof(unsigned long), GFP_KERNEL); if (!slot->roe_bitmap) - return -ENOMEM; + goto fail1; + slot->partial_roe_bitmap = kvzalloc(BITS_TO_LONGS(slot->npages) * + sizeof(unsigned long), GFP_KERNEL); + if (!slot->partial_roe_bitmap) + goto fail2; + slot->prot_list = kvzalloc(sizeof(struct list_head), GFP_KERNEL); + if (!slot->prot_list) + goto fail3; + INIT_LIST_HEAD(slot->prot_list); return 0; +fail3: + kvfree(slot->partial_roe_bitmap); +fail2: + kvfree(slot->roe_bitmap); +fail1: + return -ENOMEM; + +} + +static bool kvm_roe_protected_range(struct kvm_memory_slot *slot, gpa_t gpa, + int len) +{ + struct list_head *pos; + struct protected_chunk *cur_chunk; + + list_for_each(pos, slot->prot_list) { + cur_chunk = list_entry(pos, struct protected_chunk, list); + if (kvm_roe_range_overlap(cur_chunk, gpa, len)) + return true; + } + return false; +} + +bool kvm_roe_check_range(struct kvm_memory_slot *slot, gfn_t gfn, int offset, + int len) +{ + gpa_t gpa = (gfn << PAGE_SHIFT) + offset; + if (!gfn_is_partial_roe(slot, gfn)) + return false; + return kvm_roe_protected_range(slot, gpa, len); } + void kvm_roe_free(struct kvm_memory_slot *slot) { + struct protected_chunk *pos, *n; + struct list_head *head = slot->prot_list; + kvfree(slot->roe_bitmap); + kvfree(slot->partial_roe_bitmap); + list_for_each_entry_safe(pos, n, head, list) { + list_del(&pos->list); + kvfree(pos); + } + kvfree(slot->prot_list); } static void kvm_roe_protect_slot(struct kvm *kvm, struct kvm_memory_slot *slot, - gfn_t gfn, u64 npages) + gfn_t gfn, u64 npages, bool partial) { int i; + void *bitmap; + if (partial) + bitmap = slot->partial_roe_bitmap; + else + bitmap = slot->roe_bitmap; for (i = gfn - slot->base_gfn; i < gfn + npages - slot->base_gfn; i++) - set_bit(i, slot->roe_bitmap); + set_bit(i, bitmap); kvm_roe_arch_commit_protection(kvm, slot); } -static int __kvm_roe_protect_range(struct kvm *kvm, gpa_t gpa, u64 npages) +static int __kvm_roe_protect_range(struct kvm *kvm, gpa_t gpa, u64 npages, + bool partial) { struct kvm_memory_slot *slot; gfn_t gfn = gpa >> PAGE_SHIFT; @@ -54,12 +109,12 @@ static int __kvm_roe_protect_range(struct kvm *kvm, gpa_t gpa, u64 npages) if (gfn + npages > slot->base_gfn + slot->npages) { u64 _npages = slot->base_gfn + slot->npages - gfn; - kvm_roe_protect_slot(kvm, slot, gfn, _npages); + kvm_roe_protect_slot(kvm, slot, gfn, _npages, partial); gfn += _npages; count += _npages; npages -= _npages; } else { - kvm_roe_protect_slot(kvm, slot, gfn, npages); + kvm_roe_protect_slot(kvm, slot, gfn, npages, partial); count += npages; npages = 0; } @@ -69,12 +124,13 @@ static int __kvm_roe_protect_range(struct kvm *kvm, gpa_t gpa, u64 npages) return count; } -static int kvm_roe_protect_range(struct kvm *kvm, gpa_t gpa, u64 npages) +static int kvm_roe_protect_range(struct kvm *kvm, gpa_t gpa, u64 npages, + bool partial) { int r; mutex_lock(&kvm->slots_lock); - r = __kvm_roe_protect_range(kvm, gpa, npages); + r = __kvm_roe_protect_range(kvm, gpa, npages, partial); mutex_unlock(&kvm->slots_lock); return r; } @@ -103,7 +159,7 @@ static int kvm_roe_full_protect_range(struct kvm_vcpu *vcpu, u64 gva, continue; if (!access_ok(VERIFY_WRITE, hva, 1 << PAGE_SHIFT)) continue; - status = kvm_roe_protect_range(vcpu->kvm, gpa, 1); + status = kvm_roe_protect_range(vcpu->kvm, gpa, 1, false); if (status > 0) count += status; } @@ -112,6 +168,139 @@ static int kvm_roe_full_protect_range(struct kvm_vcpu *vcpu, u64 gva, return count; } +static int kvm_roe_insert_chunk_next(struct list_head *pos, u64 gpa, u64 size) +{ + struct protected_chunk *chunk; + + chunk = kvzalloc(sizeof(struct protected_chunk), GFP_KERNEL); + chunk->gpa = gpa; + chunk->size = size; + INIT_LIST_HEAD(&chunk->list); + list_add(&chunk->list, pos); + return size; +} + +static int kvm_roe_expand_chunk(struct protected_chunk *pos, u64 gpa, u64 size) +{ + u64 old_ptr = pos->gpa; + u64 old_size = pos->size; + + if (gpa < old_ptr) + pos->gpa = gpa; + if (gpa + size > old_ptr + old_size) + pos->size = gpa + size - pos->gpa; + return size; +} + +static bool kvm_roe_merge_chunks(struct protected_chunk *chunk) +{ + /*attempt merging 2 consecutive given the first one*/ + struct protected_chunk *next = list_next_entry(chunk, list); + + if (!kvm_roe_range_overlap(chunk, next->gpa, next->size)) + return false; + kvm_roe_expand_chunk(chunk, next->gpa, next->size); + list_del(&next->list); + kvfree(next); + return true; +} + +static int __kvm_roe_insert_chunk(struct kvm_memory_slot *slot, u64 gpa, + u64 size) +{ + /* kvm->slots_lock must be acquired*/ + struct protected_chunk *pos; + struct list_head *head = slot->prot_list; + + if (list_empty(head)) + return kvm_roe_insert_chunk_next(head, gpa, size); + /* + * pos here will never get deleted maybe the next one will + * that is why list_for_each_entry_safe is completely unsafe + */ + list_for_each_entry(pos, head, list) { + if (kvm_roe_range_overlap(pos, gpa, size)) { + int ret = kvm_roe_expand_chunk(pos, gpa, size); + + while (head != pos->list.next) + if (!kvm_roe_merge_chunks(pos)) + break; + return ret; + } + if (pos->gpa > gpa) { + struct protected_chunk *prev; + + prev = list_prev_entry(pos, list); + return kvm_roe_insert_chunk_next(&prev->list, gpa, + size); + } + } + pos = list_last_entry(head, struct protected_chunk, list); + + return kvm_roe_insert_chunk_next(&pos->list, gpa, size); +} + +static int kvm_roe_insert_chunk(struct kvm *kvm, u64 gpa, u64 size) +{ + struct kvm_memory_slot *slot; + gfn_t gfn = gpa >> PAGE_SHIFT; + int ret; + + mutex_lock(&kvm->slots_lock); + slot = gfn_to_memslot(kvm, gfn); + ret = __kvm_roe_insert_chunk(slot, gpa, size); + mutex_unlock(&kvm->slots_lock); + return ret; +} + +static int kvm_roe_partial_page_protect(struct kvm_vcpu *vcpu, u64 gva, + u64 size) +{ + gpa_t gpa = kvm_mmu_gva_to_gpa_system(vcpu, gva, NULL); + + kvm_roe_protect_range(vcpu->kvm, gpa, 1, true); + return kvm_roe_insert_chunk(vcpu->kvm, gpa, size); +} + +static int kvm_roe_partial_protect(struct kvm_vcpu *vcpu, u64 gva, u64 size) +{ + u64 gva_start = gva; + u64 gva_end = gva+size; + u64 gpn_start = gva_start >> PAGE_SHIFT; + u64 gpn_end = gva_end >> PAGE_SHIFT; + u64 _size; + int count = 0; + // We need to make sure that there will be no overflow or zero size + if (gva_end <= gva_start) + return -EINVAL; + + // protect the partial page at the start + if (gpn_end > gpn_start) + _size = PAGE_SIZE - (gva_start & PAGE_MASK) + 1; + else + _size = size; + size -= _size; + count += kvm_roe_partial_page_protect(vcpu, gva_start, _size); + // full protect in the middle pages + if (gpn_end - gpn_start > 1) { + int ret; + u64 _gva = (gpn_start + 1) << PAGE_SHIFT; + u64 npages = gpn_end - gpn_start - 1; + + size -= npages << PAGE_SHIFT; + ret = kvm_roe_full_protect_range(vcpu, _gva, npages); + if (ret > 0) + count += ret << PAGE_SHIFT; + } + // protect the partial page at the end + if (size != 0) + count += kvm_roe_partial_page_protect(vcpu, + gpn_end << PAGE_SHIFT, size); + if (count == 0) + return -EINVAL; + return count; +} + int kvm_roe(struct kvm_vcpu *vcpu, u64 a0, u64 a1, u64 a2, u64 a3) { int ret; @@ -123,11 +312,14 @@ int kvm_roe(struct kvm_vcpu *vcpu, u64 a0, u64 a1, u64 a2, u64 a3) return -KVM_ENOSYS; switch (a0) { case ROE_VERSION: - ret = 1; //current version + ret = 2; //current version break; case ROE_MPROTECT: ret = kvm_roe_full_protect_range(vcpu, a1, a2); break; + case ROE_MPROTECT_CHUNK: + ret = kvm_roe_partial_protect(vcpu, a1, a2); + break; default: ret = -EINVAL; } diff --git a/virt/kvm/roe_generic.h b/virt/kvm/roe_generic.h index 42d6ad3a4971..006fc7b52bba 100644 --- a/virt/kvm/roe_generic.h +++ b/virt/kvm/roe_generic.h @@ -13,10 +13,16 @@ void kvm_roe_free(struct kvm_memory_slot *slot); int kvm_roe_init(struct kvm_memory_slot *slot); +bool kvm_roe_check_range(struct kvm_memory_slot *slot, gfn_t gfn, int offset, + int len); static inline bool gfn_is_full_roe(struct kvm_memory_slot *slot, gfn_t gfn) { return test_bit(gfn - slot->base_gfn, slot->roe_bitmap); } +static inline bool gfn_is_partial_roe(struct kvm_memory_slot *slot, gfn_t gfn) +{ + return test_bit(gfn - slot->base_gfn, slot->partial_roe_bitmap); +} #else static void kvm_roe_free(struct kvm_memory_slot *slot) {} static int kvm_roe_init(struct kvm_memory_slot *slot) { return 0; } @@ -24,6 +30,15 @@ static inline bool gfn_is_full_roe(struct kvm_memory_slot *slot, gfn_t gfn) { return false; } +static inline bool gfn_is_partial_roe(struct kvm_memory_slot *slot, gfn_t gfn) +{ + return false; +} +static bool kvm_roe_check_range(struct kvm_memory_slot *slot, gfn_t gfn, + int offset, int len) +{ + return false; +} #endif #endif From patchwork Sun Nov 4 17:11:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ahmed Soliman X-Patchwork-Id: 10666961 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0142A13A4 for ; Sun, 4 Nov 2018 17:13:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DC370289F4 for ; Sun, 4 Nov 2018 17:13:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CDD4228A57; Sun, 4 Nov 2018 17:13:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id 19E8F289F4 for ; Sun, 4 Nov 2018 17:13:43 +0000 (UTC) Received: (qmail 32339 invoked by uid 550); 4 Nov 2018 17:12:45 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 32284 invoked from network); 4 Nov 2018 17:12:44 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=DwaIjaOplxzcZ7ftx2a6kgsWPHMYRimWe+wNdR5/hJE=; b=MhkdCSshEjRNx9P6uI8UFpbF8xhZBi+BXrVUGq0azBHVfb+d1zjeBVZE0MZmNYcXRY MA6om+WJgl40o+v+zhnVpCAqEXp+JlmQvMJ4Xsll1pSAkOtANQ6qWh/Q+ckP7nV1hXa6 CqzPOGiKXUiwojtMSBT3idTVZQFRgNoqAw4wZ8VeT65adKtmaINQAET4gwQnH/RF30w3 f+cnlNNaERkdOCwvSvDkYWwImy5CnYaVJmX6iV8E9cqT+RePOhDQZSqKCdVOQeHKWsQn AngBM2mt/BjtcyGnUowd/4FXyQ+gF42PKnufo74QdGHHdrfViUkkkv9gWRhFm4I0zBtW 4GyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=DwaIjaOplxzcZ7ftx2a6kgsWPHMYRimWe+wNdR5/hJE=; b=JZPvN/WOmp90WV2U8nEPpOAACAbjg5zKq/yeYeh9yWw7j7bleGqapOMCuPiSdKI2t7 l69+IVl9BTwXA5LgShdiXW4LH14vzuJ5eMbh9GtOWP5AsbDb/er1bBZVClzqW8Wg6wud KqFEUtoDizR1ZtXcj2i5uOxc+6f6cMDDziqCDmBgwczF30mKOhmh8SRKu1UOMEhVSEK+ T13rtyUzT4l1i5ECztg1sQjCYwoaHoUiadNmnCFTh6XWWL+50UPhiVBtc8e31ioN3fME aSExdSu7Zuftl00RSCAyd13z+8lmw+CDNSo7844TjCa3qoJbmwJFdGLt70pq6bW7j9v+ qNjw== X-Gm-Message-State: AGRZ1gKHjXJKLc7UN0GMXsSBm0dRFxFFjyEmLCB0MVyvYrZskRCKs4bS ABvueyyFY88DImjN7gpSPs8= X-Google-Smtp-Source: AJdET5e89ZUGWapFPY6etHj5e1LT86pqcOptFq7O4qxay+gQJ0F3xdZD6d3MGP1AJX6TZZjit4uaxQ== X-Received: by 2002:adf:f148:: with SMTP id y8-v6mr4983695wro.303.1541351553120; Sun, 04 Nov 2018 09:12:33 -0800 (PST) From: Ahmed Abd El Mawgood To: Paolo Bonzini , rkrcmar@redhat.com, Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , hpa@zytor.com, x86@kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, ahmedsoliman0x666@gmail.com, ovich00@gmail.com, kernel-hardening@lists.openwall.com, nigel.edwards@hpe.com, Boris Lukashev , Hossam Hassan <7ossam9063@gmail.com>, "Ahmed Lotfy igor . stoppa @ gmail . com" Subject: [PATCH V6 7/8] KVM: X86: Port ROE_MPROTECT_CHUNK to x86 Date: Sun, 4 Nov 2018 19:11:23 +0200 Message-Id: <20181104171124.5717-8-ahmedsoliman0x666@gmail.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> References: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> X-Virus-Scanned: ClamAV using ClamSMTP Apply d->memslot->partial_roe_bitmap to shadow page table entries too. Signed-off-by: Ahmed Abd El Mawgood --- arch/x86/kvm/roe.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/roe.c b/arch/x86/kvm/roe.c index cd3e6944c15f..b2b50fbcd598 100644 --- a/arch/x86/kvm/roe.c +++ b/arch/x86/kvm/roe.c @@ -25,9 +25,12 @@ static bool __rmap_write_protect_roe(struct kvm *kvm, struct rmap_iterator iter; bool prot; bool flush = false; + void *full_bmp = d->memslot->roe_bitmap; + void *part_bmp = d->memslot->partial_roe_bitmap; for_each_rmap_spte(rmap_head, &iter, sptep) { - prot = !test_bit(d->i, d->memslot->roe_bitmap) && pt_protect; + prot = !(test_bit(d->i, full_bmp) || test_bit(d->i, part_bmp)); + prot = prot && pt_protect; flush |= spte_write_protect(sptep, prot); d->i++; } From patchwork Sun Nov 4 17:11:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ahmed Soliman X-Patchwork-Id: 10666963 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B2D6F13A4 for ; Sun, 4 Nov 2018 17:13:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 965A928C82 for ; Sun, 4 Nov 2018 17:13:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 86CA1290B8; Sun, 4 Nov 2018 17:13:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id B145C28C82 for ; Sun, 4 Nov 2018 17:13:56 +0000 (UTC) Received: (qmail 32545 invoked by uid 550); 4 Nov 2018 17:12:47 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 32478 invoked from network); 4 Nov 2018 17:12:47 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=TGhpIguxZKqGE+GdUaatDC/UNPrNnEL4NBgLd2GI3no=; b=t/EGvrw1MpSeR5jsmJkKVZg/m5oUFjSYOS8yLE8qz4cIL3Gcbn6/iPRjhO5O+lnmSU nyZzJwr3j5F7HdkUS6YNQf1+53OZg5+8kIC+4SL8JQzJZDitH1DCBzYlDFMQbnfFDUNc UP9aMK5MBCWXWGBkPIToeWyYPU2b58sCramhc7ZshzaAVB3gW5qS75TqutCV0qVJwjxr m8FqotY01NXAfi4AFJHJVrZ7YOawRuIEdVDNfuRpyI1evVjJqVxYoxaUkmF5nT6SDB+o dFc7PskmcOKqsW1RHd3pQL2cdhDUSEqwVyFUPKsl8g3oGQGC1x99w7QWqO4Ye0zsKEk+ zqdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=TGhpIguxZKqGE+GdUaatDC/UNPrNnEL4NBgLd2GI3no=; b=ikhFszKI08WQMEasEDycRFKdbJlGVcT7Ddr6DmEgiCqSyIzuOpWx/APr/ovBQ4O4bK SJ1JlWRbQsh5YskgjY+m52dZTUBmaNmsNHst8y8p1ySrIfkKKgmP2IoS7xjyt4SDRMdI KNk7VKAmBnaFiGyIF5jEn7OTbBPjouvsbWZHwF1k9NMN78ylk/D3ZvdMi8PMWowdZT9T qXKsjTc0oGooaf0n9uwo/rbzMyz1H7OR8CJ8rH6BhgW4yGlfhKplCWVKr0OTTpsESX5O QvrnmieJmrfJb5ctEZoR9dZNqNjy+0n2uo6HVCJfbr2AcA/55CfSkLhDdHXH2C8n82cB 0xOQ== X-Gm-Message-State: AGRZ1gL9d4oE6AxixsjKimK4j40Rm3/X/LT3NijdYa0Ckxs74OgHflLG 6V/losbjsE5ZP0Tx3rDE+Rk= X-Google-Smtp-Source: AJdET5ejay/sWk71bozJheRrobQzLIg9jrGDQJAiR0ikmiurOlkCuZvlKrsGOAnug1qWJDWykMx8gA== X-Received: by 2002:a1c:578c:: with SMTP id l134-v6mr3895086wmb.135.1541351555736; Sun, 04 Nov 2018 09:12:35 -0800 (PST) From: Ahmed Abd El Mawgood To: Paolo Bonzini , rkrcmar@redhat.com, Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , hpa@zytor.com, x86@kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, ahmedsoliman0x666@gmail.com, ovich00@gmail.com, kernel-hardening@lists.openwall.com, nigel.edwards@hpe.com, Boris Lukashev , Hossam Hassan <7ossam9063@gmail.com>, "Ahmed Lotfy igor . stoppa @ gmail . com" Subject: [PATCH V6 8/8] KVM: Log ROE violations in system log Date: Sun, 4 Nov 2018 19:11:24 +0200 Message-Id: <20181104171124.5717-9-ahmedsoliman0x666@gmail.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> References: <20181104171124.5717-1-ahmedsoliman0x666@gmail.com> X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Ahmed Abd El Mawgood --- virt/kvm/kvm_main.c | 7 +++++++ virt/kvm/roe.c | 14 ++++++++++++++ virt/kvm/roe_generic.h | 2 ++ 3 files changed, 23 insertions(+) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 48c5d9d9474e..d625db7f5350 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -63,6 +63,8 @@ #include "vfio.h" #include "roe_generic.h" +#include + #define CREATE_TRACE_POINTS #include @@ -1867,6 +1869,7 @@ static u64 roe_gfn_to_hva(struct kvm_memory_slot *slot, gfn_t gfn, int offset, addr = __gfn_to_hva_many(slot, gfn, NULL, false); return addr; } + static int __kvm_write_guest_page(struct kvm_memory_slot *memslot, gfn_t gfn, const void *data, int offset, int len) { @@ -1874,6 +1877,10 @@ static int __kvm_write_guest_page(struct kvm_memory_slot *memslot, gfn_t gfn, unsigned long addr; addr = roe_gfn_to_hva(memslot, gfn, offset, len); + if (gfn_is_full_roe(memslot, gfn) || + kvm_roe_check_range(memslot, gfn, offset, len)) + kvm_warning_roe_violation((gfn << PAGE_SHIFT) + offset, data, + len); if (kvm_is_error_hva(addr)) return -EFAULT; r = __copy_to_user((void __user *)addr + offset, data, len); diff --git a/virt/kvm/roe.c b/virt/kvm/roe.c index e94314fed3a3..c30c6b028638 100644 --- a/virt/kvm/roe.c +++ b/virt/kvm/roe.c @@ -76,6 +76,20 @@ void kvm_roe_free(struct kvm_memory_slot *slot) kvfree(slot->prot_list); } +void kvm_warning_roe_violation(u64 addr, const void *data, int len) +{ + int i; + const char *d = data; + char *buf = kvmalloc(len * 3 + 1, GFP_KERNEL); + + for (i = 0; i < len; i++) + sprintf(buf+3*i, " %02x", d[i]); + pr_warn("ROE violation:\n"); + pr_warn("\tAttempt to write %d bytes at address 0x%08llx\n", len, addr); + pr_warn("\tData: %s\n", buf); + kvfree(buf); +} + static void kvm_roe_protect_slot(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, u64 npages, bool partial) { diff --git a/virt/kvm/roe_generic.h b/virt/kvm/roe_generic.h index 006fc7b52bba..bce426441468 100644 --- a/virt/kvm/roe_generic.h +++ b/virt/kvm/roe_generic.h @@ -11,6 +11,7 @@ */ #ifdef CONFIG_KVM_ROE +void kvm_warning_roe_violation(u64 addr, const void *data, int len); void kvm_roe_free(struct kvm_memory_slot *slot); int kvm_roe_init(struct kvm_memory_slot *slot); bool kvm_roe_check_range(struct kvm_memory_slot *slot, gfn_t gfn, int offset, @@ -39,6 +40,7 @@ static bool kvm_roe_check_range(struct kvm_memory_slot *slot, gfn_t gfn, { return false; } +static void kvm_warning_roe_violation(u64 addr, const void *data, int len) {} #endif #endif