From patchwork Fri Jan 31 16:42:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11360273 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 111F0139A for ; Fri, 31 Jan 2020 16:43:44 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EA9E8206F0 for ; Fri, 31 Jan 2020 16:43:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EA9E8206F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZNC-0008BX-UH; Fri, 31 Jan 2020 16:42:26 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZNC-0008BS-3O for xen-devel@lists.xenproject.org; Fri, 31 Jan 2020 16:42:26 +0000 X-Inumbo-ID: a88dd706-4448-11ea-8bec-12813bfff9fa Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id a88dd706-4448-11ea-8bec-12813bfff9fa; Fri, 31 Jan 2020 16:42:25 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id B5565AD3C; Fri, 31 Jan 2020 16:42:23 +0000 (UTC) From: Jan Beulich To: "xen-devel@lists.xenproject.org" References: Message-ID: Date: Fri, 31 Jan 2020 17:42:25 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.4.2 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Subject: [Xen-devel] [PATCH v4 1/7] SVM: drop asm/hvm/emulate.h inclusion from vmcb.h X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Petre Pircalabu , Kevin Tian , Tamas K Lengyel , Wei Liu , Paul Durrant , George Dunlap , Andrew Cooper , Tim Deegan , Jun Nakajima , Alexandru Isaila , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" It's not needed there and introduces a needless, almost global dependency. Include the file (or in some cases just xen/err.h) where actually needed, or - in one case - simply forward-declare a struct. In microcode*.c take the opportunity and also re-order a few other #include-s. Signed-off-by: Jan Beulich Acked-by: Andrew Cooper Reviewed-by: Alexandru Isaila Reviewed-by: Paul Durrant Reviewed-by: Kevin Tian --- v4: New. --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -55,6 +55,7 @@ #include #include #include +#include #include #include #include --- a/xen/arch/x86/hvm/ioreq.c +++ b/xen/arch/x86/hvm/ioreq.c @@ -28,6 +28,7 @@ #include #include +#include #include #include #include --- a/xen/arch/x86/hvm/svm/emulate.c +++ b/xen/arch/x86/hvm/svm/emulate.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include --- a/xen/arch/x86/hvm/vm_event.c +++ b/xen/arch/x86/hvm/vm_event.c @@ -22,6 +22,7 @@ #include #include +#include #include #include --- a/xen/arch/x86/microcode.c +++ b/xen/arch/x86/microcode.c @@ -22,9 +22,10 @@ */ #include -#include -#include +#include #include +#include +#include #include #include #include --- a/xen/arch/x86/microcode_amd.c +++ b/xen/arch/x86/microcode_amd.c @@ -14,9 +14,10 @@ * License version 2. See file COPYING for details. */ -#include -#include +#include #include +#include +#include #include #include #include --- a/xen/arch/x86/microcode_intel.c +++ b/xen/arch/x86/microcode_intel.c @@ -21,9 +21,10 @@ * 2 of the License, or (at your option) any later version. */ -#include -#include +#include #include +#include +#include #include #include #include --- a/xen/arch/x86/mm/shadow/hvm.c +++ b/xen/arch/x86/mm/shadow/hvm.c @@ -28,6 +28,7 @@ #include #include +#include #include #include "private.h" --- a/xen/arch/x86/pv/emul-gate-op.c +++ b/xen/arch/x86/pv/emul-gate-op.c @@ -19,6 +19,7 @@ * along with this program; If not, see . */ +#include #include #include #include --- a/xen/include/asm-x86/hvm/svm/vmcb.h +++ b/xen/include/asm-x86/hvm/svm/vmcb.h @@ -20,8 +20,6 @@ #define __ASM_X86_HVM_SVM_VMCB_H__ #include -#include - /* general 1 intercepts */ enum GenericIntercept1bits --- a/xen/include/asm-x86/hvm/vmx/vmx.h +++ b/xen/include/asm-x86/hvm/vmx/vmx.h @@ -97,6 +97,7 @@ void vmx_asm_do_vmentry(void); void vmx_intr_assist(void); void noreturn vmx_do_resume(struct vcpu *); void vmx_vlapic_msr_changed(struct vcpu *v); +struct hvm_emulate_ctxt; void vmx_realmode_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt); void vmx_realmode(struct cpu_user_regs *regs); void vmx_update_debug_state(struct vcpu *v); From patchwork Fri Jan 31 16:42:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11360275 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8113A139A for ; Fri, 31 Jan 2020 16:43:55 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 67BB1206F0 for ; Fri, 31 Jan 2020 16:43:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 67BB1206F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZNk-0008El-9Q; Fri, 31 Jan 2020 16:43:00 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZNi-0008Ea-M8 for xen-devel@lists.xenproject.org; Fri, 31 Jan 2020 16:42:58 +0000 X-Inumbo-ID: bc50be5c-4448-11ea-8bec-12813bfff9fa Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id bc50be5c-4448-11ea-8bec-12813bfff9fa; Fri, 31 Jan 2020 16:42:58 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 57D92AF27; Fri, 31 Jan 2020 16:42:57 +0000 (UTC) From: Jan Beulich To: "xen-devel@lists.xenproject.org" References: Message-ID: Date: Fri, 31 Jan 2020 17:42:59 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.4.2 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Subject: [Xen-devel] [PATCH v4 2/7] x86/HVM: rename a variable in __hvm_copy() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: George Dunlap , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Paul Durrant Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" This is to reflect its actual purpose. Also use in a 2nd place. Signed-off-by: Jan Beulich Reviewed-by: Andrew Cooper --- v4: New. --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -3255,9 +3255,9 @@ static enum hvm_translation_result __hvm while ( todo > 0 ) { enum hvm_translation_result res; - paddr_t gpa = addr & ~PAGE_MASK; + unsigned int pgoff = addr & ~PAGE_MASK; - count = min_t(int, PAGE_SIZE - gpa, todo); + count = min_t(int, PAGE_SIZE - pgoff, todo); res = hvm_translate_get_page(v, addr, flags & HVMCOPY_linear, pfec, pfinfo, &page, &gfn, &p2mt); @@ -3279,7 +3279,7 @@ static enum hvm_translation_result __hvm return HVMTRANS_need_retry; } - p = (char *)__map_domain_page(page) + (addr & ~PAGE_MASK); + p = __map_domain_page(page) + pgoff; if ( flags & HVMCOPY_to_guest ) { From patchwork Fri Jan 31 16:43:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11360277 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D509D1395 for ; Fri, 31 Jan 2020 16:44:12 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BB157206F0 for ; Fri, 31 Jan 2020 16:44:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BB157206F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZO9-0008ID-KX; Fri, 31 Jan 2020 16:43:25 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZO7-0008Hz-Mx for xen-devel@lists.xenproject.org; Fri, 31 Jan 2020 16:43:23 +0000 X-Inumbo-ID: cb343e6c-4448-11ea-b211-bc764e2007e4 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id cb343e6c-4448-11ea-b211-bc764e2007e4; Fri, 31 Jan 2020 16:43:23 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 523F6AD3C; Fri, 31 Jan 2020 16:43:22 +0000 (UTC) From: Jan Beulich To: "xen-devel@lists.xenproject.org" References: Message-ID: <90e7ed4e-aff5-1b0b-e3a8-fbb4c058a7d1@suse.com> Date: Fri, 31 Jan 2020 17:43:25 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.4.2 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Subject: [Xen-devel] [PATCH v4 3/7] x86/HVM: introduce "curr" into hvmemul_rep_{mov, sto}s() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: George Dunlap , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Paul Durrant Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" There are a number of uses of "current" already, and more may appear down the road. Latch into a local variable. At this occasion also drop stray casts from code getting touched anyway. Signed-off-by: Jan Beulich Reviewed-by: Andrew Cooper Reviewed-by: Paul Durrant --- v4: New. --- a/xen/arch/x86/hvm/emulate.c +++ b/xen/arch/x86/hvm/emulate.c @@ -1747,7 +1747,8 @@ static int hvmemul_rep_movs( { struct hvm_emulate_ctxt *hvmemul_ctxt = container_of(ctxt, struct hvm_emulate_ctxt, ctxt); - struct hvm_vcpu_io *vio = ¤t->arch.hvm.hvm_io; + struct vcpu *curr = current; + struct hvm_vcpu_io *vio = &curr->arch.hvm.hvm_io; unsigned long saddr, daddr, bytes; paddr_t sgpa, dgpa; uint32_t pfec = PFEC_page_present; @@ -1807,8 +1808,8 @@ static int hvmemul_rep_movs( } /* Check for MMIO ops */ - (void) get_gfn_query_unlocked(current->domain, sgpa >> PAGE_SHIFT, &sp2mt); - (void) get_gfn_query_unlocked(current->domain, dgpa >> PAGE_SHIFT, &dp2mt); + get_gfn_query_unlocked(curr->domain, sgpa >> PAGE_SHIFT, &sp2mt); + get_gfn_query_unlocked(curr->domain, dgpa >> PAGE_SHIFT, &dp2mt); if ( sp2mt == p2m_mmio_direct || dp2mt == p2m_mmio_direct || (sp2mt == p2m_mmio_dm && dp2mt == p2m_mmio_dm) ) @@ -1873,7 +1874,7 @@ static int hvmemul_rep_movs( rc = hvm_copy_from_guest_phys(buf, sgpa, bytes); if ( rc == HVMTRANS_okay ) - rc = hvm_copy_to_guest_phys(dgpa, buf, bytes, current); + rc = hvm_copy_to_guest_phys(dgpa, buf, bytes, curr); xfree(buf); @@ -1910,7 +1911,8 @@ static int hvmemul_rep_stos( { struct hvm_emulate_ctxt *hvmemul_ctxt = container_of(ctxt, struct hvm_emulate_ctxt, ctxt); - struct hvm_vcpu_io *vio = ¤t->arch.hvm.hvm_io; + struct vcpu *curr = current; + struct hvm_vcpu_io *vio = &curr->arch.hvm.hvm_io; unsigned long addr, bytes; paddr_t gpa; p2m_type_t p2mt; @@ -1943,7 +1945,7 @@ static int hvmemul_rep_stos( } /* Check for MMIO op */ - (void)get_gfn_query_unlocked(current->domain, gpa >> PAGE_SHIFT, &p2mt); + get_gfn_query_unlocked(curr->domain, gpa >> PAGE_SHIFT, &p2mt); switch ( p2mt ) { @@ -1992,7 +1994,7 @@ static int hvmemul_rep_stos( if ( df ) gpa -= bytes - bytes_per_rep; - rc = hvm_copy_to_guest_phys(gpa, buf, bytes, current); + rc = hvm_copy_to_guest_phys(gpa, buf, bytes, curr); if ( buf != p_data ) xfree(buf); From patchwork Fri Jan 31 16:44:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11360279 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B76A13A4 for ; Fri, 31 Jan 2020 16:45:15 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 56363206F0 for ; Fri, 31 Jan 2020 16:45:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 56363206F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZP5-0008Sw-6Q; Fri, 31 Jan 2020 16:44:23 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZP4-0008Sj-K5 for xen-devel@lists.xenproject.org; Fri, 31 Jan 2020 16:44:22 +0000 X-Inumbo-ID: eddf2364-4448-11ea-8396-bc764e2007e4 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id eddf2364-4448-11ea-8396-bc764e2007e4; Fri, 31 Jan 2020 16:44:21 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 71CA8AD3C; Fri, 31 Jan 2020 16:44:20 +0000 (UTC) From: Jan Beulich To: "xen-devel@lists.xenproject.org" References: Message-ID: Date: Fri, 31 Jan 2020 17:44:23 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.4.2 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Subject: [Xen-devel] [PATCH v4 4/7] x86/HVM: implement memory read caching for insn emulation X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Kevin Tian , Wei Liu , Paul Durrant , George Dunlap , Andrew Cooper , Jun Nakajima , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Emulation requiring device model assistance uses a form of instruction re-execution, assuming that the second (and any further) pass takes exactly the same path. This is a valid assumption as far as use of CPU registers goes (as those can't change without any other instruction executing in between), but is wrong for memory accesses. In particular it has been observed that Windows might page out buffers underneath an instruction currently under emulation (hitting between two passes). If the first pass read a memory operand successfully, any subsequent pass needs to get to see the exact same value. Introduce a cache to make sure above described assumption holds. This is a very simplistic implementation for now: Only exact matches are satisfied (no overlaps or partial reads or anything); this is sufficient for the immediate purpose of making re-execution an exact replay. The cache also won't be used just yet for guest page walks; that'll be the subject of a subsequent change. With the cache being generally transparent to upper layers, but with it having limited capacity yet being required for correctness, certain users of hvm_copy_from_guest_*() need to disable caching temporarily, without invalidating the cache. Note that the adjustments here to hvm_hypercall() and hvm_task_switch() are benign at this point; they'll become relevant once we start to be able to emulate respective insns through the main emulator (and more changes will then likely be needed to nested code). As to the actual data page in this scenario, there are a couple of aspects to take into consideration: - We must be talking about an insn accessing two locations (two memory ones, one of which is MMIO, or a memory and an I/O one). - If the non I/O / MMIO side is being read, the re-read (if it occurs at all) is having its result discarded, by taking the shortcut through the first switch()'s STATE_IORESP_READY case in hvmemul_do_io(). Note how, among all the re-issue sanity checks there, we avoid comparing the actual data. - If the non I/O / MMIO side is being written, it is the OSes responsibility to avoid actually moving page contents to disk while there might still be a write access in flight - this is no different in behavior from bare hardware. - Read-modify-write accesses are, as always, complicated, and while we deal with them better nowadays than we did in the past, we're still not quite there to guarantee hardware like behavior in all cases anyway. Nothing is getting worse by the changes made here, afaict. In __hvm_copy() also reduce p's scope and change its type to void *. Signed-off-by: Jan Beulich --- TBD: In principle the caching here yields unnecessary the one used for insn bytes (vio->mmio_insn{,_bytes}. However, to seed the cache with the data SVM may have made available, we'd have to also know the corresponding GPA. It's not safe, however, to re-walk the page tables to find out, as the page tables may have changed in the meantime. Therefore I guess we need to keep the duplicate functionality for now. A possible solution to this could be to use a physical-address-based cache for page table accesses (and looking forward also e.g. SVM/VMX insn emulation), and a linear-address- based one for all other reads. --- v4: Re-write for cache to become transparent to callers. v3: Add text about the actual data page to the description. v2: Re-base. --- a/xen/arch/x86/hvm/emulate.c +++ b/xen/arch/x86/hvm/emulate.c @@ -28,6 +28,17 @@ #include #include +struct hvmemul_cache +{ + unsigned int num_ents; + unsigned int max_ents; + struct { + paddr_t gpa:PADDR_BITS; + unsigned int size:BITS_PER_LONG - PADDR_BITS; + unsigned long data; + } ents[]; +}; + static void hvmtrace_io_assist(const ioreq_t *p) { unsigned int size, event; @@ -1866,12 +1877,17 @@ static int hvmemul_rep_movs( rc = HVMTRANS_okay; } else + { + unsigned int token = hvmemul_cache_disable(curr); + /* * We do a modicum of checking here, just for paranoia's sake and to * definitely avoid copying an unitialised buffer into guest address * space. */ rc = hvm_copy_from_guest_phys(buf, sgpa, bytes); + hvmemul_cache_restore(curr, token); + } if ( rc == HVMTRANS_okay ) rc = hvm_copy_to_guest_phys(dgpa, buf, bytes, curr); @@ -2534,6 +2550,19 @@ static int _hvm_emulate_one(struct hvm_e struct hvm_vcpu_io *vio = &curr->arch.hvm.hvm_io; int rc; + /* + * Enable caching if it's currently disabled, but leave the cache + * untouched if it's already enabled, for re-execution to consume + * entries populated by an earlier pass. + */ + if ( vio->cache->num_ents > vio->cache->max_ents ) + { + ASSERT(vio->io_req.state == STATE_IOREQ_NONE); + vio->cache->num_ents = 0; + } + else + ASSERT(vio->io_req.state == STATE_IORESP_READY); + hvm_emulate_init_per_insn(hvmemul_ctxt, vio->mmio_insn, vio->mmio_insn_bytes); @@ -2547,6 +2576,7 @@ static int _hvm_emulate_one(struct hvm_e { vio->mmio_cache_count = 0; vio->mmio_insn_bytes = 0; + hvmemul_cache_disable(curr); } else { @@ -2838,6 +2868,123 @@ void hvm_dump_emulation_state(const char hvmemul_ctxt->insn_buf); } +int hvmemul_cache_init(struct vcpu *v) +{ + /* + * No insn can access more than 16 independent linear addresses (AVX512F + * scatters/gathers being the worst). Each such linear range can span a + * page boundary, i.e. may require two page walks. Account for each insn + * byte individually. + */ + const unsigned int nents = (CONFIG_PAGING_LEVELS + 1) * + (MAX_INST_LEN + 16 * 2); + struct hvmemul_cache *cache = xmalloc_flex_struct(struct hvmemul_cache, + ents, nents); + + if ( !cache ) + return -ENOMEM; + + /* Cache is disabled initially. */ + cache->num_ents = nents + 1; + cache->max_ents = nents; + + v->arch.hvm.hvm_io.cache = cache; + + return 0; +} + +unsigned int hvmemul_cache_disable(struct vcpu *v) +{ + struct hvmemul_cache *cache = v->arch.hvm.hvm_io.cache; + unsigned int token = cache->num_ents; + + cache->num_ents = cache->max_ents + 1; + + return token; +} + +void hvmemul_cache_restore(struct vcpu *v, unsigned int token) +{ + struct hvmemul_cache *cache = v->arch.hvm.hvm_io.cache; + + ASSERT(cache->num_ents > cache->max_ents); + cache->num_ents = token; +} + +bool hvmemul_read_cache(const struct vcpu *v, paddr_t gpa, + void *buffer, unsigned int size) +{ + const struct hvmemul_cache *cache = v->arch.hvm.hvm_io.cache; + unsigned int i; + + /* Cache unavailable? */ + if ( cache->num_ents > cache->max_ents ) + return false; + + while ( size > sizeof(cache->ents->data) ) + { + i = gpa & (sizeof(cache->ents->data) - 1) + ? -gpa & (sizeof(cache->ents->data) - 1) + : sizeof(cache->ents->data); + if ( !hvmemul_read_cache(v, gpa, buffer, i) ) + return false; + gpa += i; + buffer += i; + size -= i; + } + + for ( i = 0; i < cache->num_ents; ++i ) + if ( cache->ents[i].gpa == gpa && cache->ents[i].size == size ) + { + memcpy(buffer, &cache->ents[i].data, size); + return true; + } + + return false; +} + +void hvmemul_write_cache(const struct vcpu *v, paddr_t gpa, + const void *buffer, unsigned int size) +{ + struct hvmemul_cache *cache = v->arch.hvm.hvm_io.cache; + unsigned int i; + + /* Cache unavailable? */ + if ( cache->num_ents > cache->max_ents ) + return; + + while ( size > sizeof(cache->ents->data) ) + { + i = gpa & (sizeof(cache->ents->data) - 1) + ? -gpa & (sizeof(cache->ents->data) - 1) + : sizeof(cache->ents->data); + hvmemul_write_cache(v, gpa, buffer, i); + gpa += i; + buffer += i; + size -= i; + } + + for ( i = 0; i < cache->num_ents; ++i ) + if ( cache->ents[i].gpa == gpa && cache->ents[i].size == size ) + { + memcpy(&cache->ents[i].data, buffer, size); + return; + } + + if ( unlikely(i >= cache->max_ents) ) + { + ASSERT_UNREACHABLE(); + return; + } + + cache->ents[i].gpa = gpa; + cache->ents[i].size = size; + + memcpy(&cache->ents[i].data, buffer, size); + + cache->num_ents = i + 1; +} + /* * Local variables: * mode: C --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -717,6 +717,8 @@ int hvm_domain_initialise(struct domain /* This function and all its descendants need to be to be idempotent. */ void hvm_domain_relinquish_resources(struct domain *d) { + struct vcpu *v; + if ( hvm_funcs.domain_relinquish_resources ) alternative_vcall(hvm_funcs.domain_relinquish_resources, d); @@ -733,6 +735,9 @@ void hvm_domain_relinquish_resources(str rtc_deinit(d); pmtimer_deinit(d); hpet_deinit(d); + + for_each_vcpu ( d, v ) + hvmemul_cache_destroy(v); } void hvm_domain_destroy(struct domain *d) @@ -1538,6 +1543,10 @@ int hvm_vcpu_initialise(struct vcpu *v) v->arch.hvm.inject_event.vector = HVM_EVENT_VECTOR_UNSET; + rc = hvmemul_cache_init(v); + if ( rc ) + goto fail4; + rc = setup_compat_arg_xlat(v); /* teardown: free_compat_arg_xlat() */ if ( rc != 0 ) goto fail4; @@ -1573,6 +1582,7 @@ int hvm_vcpu_initialise(struct vcpu *v) fail5: free_compat_arg_xlat(v); fail4: + hvmemul_cache_destroy(v); hvm_funcs.vcpu_destroy(v); fail3: vlapic_destroy(v); @@ -2934,6 +2944,7 @@ void hvm_task_switch( unsigned int eflags, new_cpl; pagefault_info_t pfinfo; int exn_raised, rc; + unsigned int token = hvmemul_cache_disable(v); struct tss32 tss; hvm_get_segment_register(v, x86_seg_gdtr, &gdt); @@ -3141,6 +3152,8 @@ void hvm_task_switch( out: hvm_unmap_entry(optss_desc); hvm_unmap_entry(nptss_desc); + + hvmemul_cache_restore(v, token); } enum hvm_translation_result hvm_translate_get_page( @@ -3231,7 +3244,6 @@ static enum hvm_translation_result __hvm gfn_t gfn; struct page_info *page; p2m_type_t p2mt; - char *p; int count, todo = size; ASSERT(is_hvm_vcpu(v)); @@ -3279,11 +3291,17 @@ static enum hvm_translation_result __hvm return HVMTRANS_need_retry; } - p = __map_domain_page(page) + pgoff; - - if ( flags & HVMCOPY_to_guest ) + if ( (flags & HVMCOPY_to_guest) || + !hvmemul_read_cache(v, gfn_to_gaddr(gfn) | pgoff, buf, count) ) { - if ( p2m_is_discard_write(p2mt) ) + void *p = __map_domain_page(page) + pgoff; + + if ( !(flags & HVMCOPY_to_guest) ) + { + memcpy(buf, p, count); + hvmemul_write_cache(v, gfn_to_gaddr(gfn) | pgoff, buf, count); + } + else if ( p2m_is_discard_write(p2mt) ) { static unsigned long lastpage; @@ -3300,13 +3318,9 @@ static enum hvm_translation_result __hvm memset(p, 0, count); paging_mark_pfn_dirty(v->domain, _pfn(gfn_x(gfn))); } - } - else - { - memcpy(buf, p, count); - } - unmap_domain_page(p); + unmap_domain_page(p); + } addr += count; if ( buf ) --- a/xen/arch/x86/hvm/hypercall.c +++ b/xen/arch/x86/hvm/hypercall.c @@ -22,6 +22,7 @@ #include #include +#include #include static long hvm_memory_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) @@ -159,6 +160,7 @@ int hvm_hypercall(struct cpu_user_regs * struct domain *currd = curr->domain; int mode = hvm_guest_x86_mode(curr); unsigned long eax = regs->eax; + unsigned int token; switch ( mode ) { @@ -183,7 +185,18 @@ int hvm_hypercall(struct cpu_user_regs * } if ( (eax & 0x80000000) && is_viridian_domain(currd) ) - return viridian_hypercall(regs); + { + int ret; + + /* See comment below. */ + token = hvmemul_cache_disable(curr); + + ret = viridian_hypercall(regs); + + hvmemul_cache_restore(curr, token); + + return ret; + } BUILD_BUG_ON(ARRAY_SIZE(hvm_hypercall_table) > ARRAY_SIZE(hypercall_args_table)); @@ -202,6 +215,12 @@ int hvm_hypercall(struct cpu_user_regs * return HVM_HCALL_completed; } + /* + * Caching is intended for instruction emulation only. Disable it + * for any accesses by hypercall argument copy-in / copy-out. + */ + token = hvmemul_cache_disable(curr); + curr->hcall_preempted = false; if ( mode == 8 ) @@ -295,6 +314,8 @@ int hvm_hypercall(struct cpu_user_regs * #endif } + hvmemul_cache_restore(curr, token); + HVM_DBG_LOG(DBG_LEVEL_HCALL, "hcall%lu -> %lx", eax, regs->rax); if ( curr->hcall_preempted ) --- a/xen/arch/x86/hvm/intercept.c +++ b/xen/arch/x86/hvm/intercept.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -163,6 +164,9 @@ int hvm_process_io_intercept(const struc { if ( p->data_is_ptr ) { + struct vcpu *curr = current; + unsigned int token = hvmemul_cache_disable(curr); + data = 0; switch ( hvm_copy_from_guest_phys(&data, p->data + step * i, p->size) ) @@ -179,9 +183,11 @@ int hvm_process_io_intercept(const struc ASSERT_UNREACHABLE(); /* fall through */ default: - domain_crash(current->domain); + domain_crash(curr->domain); return X86EMUL_UNHANDLEABLE; } + + hvmemul_cache_restore(curr, token); } else data = p->data; --- a/xen/arch/x86/hvm/svm/svm.c +++ b/xen/arch/x86/hvm/svm/svm.c @@ -1040,6 +1040,8 @@ void svm_vmenter_helper(const struct cpu struct vcpu *curr = current; struct vmcb_struct *vmcb = curr->arch.hvm.svm.vmcb; + ASSERT(hvmemul_cache_disabled(curr)); + svm_asid_handle_vmrun(); if ( unlikely(tb_init_done) ) --- a/xen/arch/x86/hvm/vmsi.c +++ b/xen/arch/x86/hvm/vmsi.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -607,6 +608,7 @@ void msix_write_completion(struct vcpu * if ( !ctrl_address && snoop_addr && v->arch.hvm.hvm_io.msix_snoop_gpa ) { + unsigned int token = hvmemul_cache_disable(v); const struct msi_desc *desc; uint32_t data; @@ -621,6 +623,8 @@ void msix_write_completion(struct vcpu * sizeof(data)) == HVMTRANS_okay && !(data & PCI_MSIX_VECTOR_BITMASK) ) ctrl_address = snoop_addr; + + hvmemul_cache_restore(v, token); } if ( !ctrl_address ) --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -4347,6 +4347,8 @@ bool vmx_vmenter_helper(const struct cpu struct hvm_vcpu_asid *p_asid; bool_t need_flush; + ASSERT(hvmemul_cache_disabled(curr)); + /* Shadow EPTP can't be updated here because irqs are disabled */ if ( nestedhvm_vcpu_in_guestmode(curr) && vcpu_nestedhvm(curr).stale_np2m ) return false; --- a/xen/include/asm-x86/hvm/emulate.h +++ b/xen/include/asm-x86/hvm/emulate.h @@ -13,6 +13,7 @@ #define __ASM_X86_HVM_EMULATE_H__ #include +#include #include #include @@ -96,6 +97,31 @@ int hvmemul_do_pio_buffer(uint16_t port, uint8_t dir, void *buffer); +#ifdef CONFIG_HVM +int __must_check hvmemul_cache_init(struct vcpu *v); +static inline void hvmemul_cache_destroy(struct vcpu *v) +{ + XFREE(v->arch.hvm.hvm_io.cache); +} +bool hvmemul_read_cache(const struct vcpu *, paddr_t gpa, + void *buffer, unsigned int size); +void hvmemul_write_cache(const struct vcpu *, paddr_t gpa, + const void *buffer, unsigned int size); +unsigned int hvmemul_cache_disable(struct vcpu *); +void hvmemul_cache_restore(struct vcpu *, unsigned int token); +/* For use in ASSERT()s only: */ +static inline bool hvmemul_cache_disabled(struct vcpu *v) +{ + return hvmemul_cache_disable(v) == hvmemul_cache_disable(v); +} +#else +static inline bool hvmemul_read_cache(const struct vcpu *v, paddr_t gpa, + void *buf, + unsigned int size) { return false; } +static inline void hvmemul_write_cache(const struct vcpu *v, paddr_t gpa, + const void *buf, unsigned int size) {} +#endif + void hvm_dump_emulation_state(const char *loglvl, const char *prefix, struct hvm_emulate_ctxt *hvmemul_ctxt, int rc); --- a/xen/include/asm-x86/hvm/vcpu.h +++ b/xen/include/asm-x86/hvm/vcpu.h @@ -77,6 +77,8 @@ struct hvm_vcpu_io { /* For retries we shouldn't re-fetch the instruction. */ unsigned int mmio_insn_bytes; unsigned char mmio_insn[16]; + struct hvmemul_cache *cache; + /* * For string instruction emulation we need to be able to signal a * necessary retry through other than function return codes. From patchwork Fri Jan 31 16:45:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11360281 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 124D1139A for ; Fri, 31 Jan 2020 16:45:52 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E189320663 for ; Fri, 31 Jan 2020 16:45:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E189320663 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZPh-0000CA-1r; Fri, 31 Jan 2020 16:45:01 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZPf-0000Bl-Ak for xen-devel@lists.xenproject.org; Fri, 31 Jan 2020 16:44:59 +0000 X-Inumbo-ID: 03d83458-4449-11ea-b211-bc764e2007e4 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 03d83458-4449-11ea-b211-bc764e2007e4; Fri, 31 Jan 2020 16:44:58 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 5052AAD3C; Fri, 31 Jan 2020 16:44:57 +0000 (UTC) From: Jan Beulich To: "xen-devel@lists.xenproject.org" References: Message-ID: <2c9ef202-a878-ab7b-5bfc-f9738d52d291@suse.com> Date: Fri, 31 Jan 2020 17:45:00 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.4.2 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Subject: [Xen-devel] [PATCH v4 5/7] x86/mm: use cache in guest_walk_tables() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Wei Liu , Paul Durrant , George Dunlap , Andrew Cooper , Tim Deegan , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Emulation requiring device model assistance uses a form of instruction re-execution, assuming that the second (and any further) pass takes exactly the same path. This is a valid assumption as far as use of CPU registers goes (as those can't change without any other instruction executing in between), but is wrong for memory accesses. In particular it has been observed that Windows might page out buffers underneath an instruction currently under emulation (hitting between two passes). If the first pass translated a linear address successfully, any subsequent pass needs to do so too, yielding the exact same translation. To guarantee this, leverage the caching that now backs HVM insn emulation. Signed-off-by: Jan Beulich --- v4: Adjust for cache now (elsewhere) being transparent to callers. Provide inline stubs for the !HVM case. v2: Don't wrongly use top_gfn for non-root gpa calculation. Re-write cache entries after setting A/D bits (an alternative would be to suppress their setting upon cache hits). --- a/xen/arch/x86/hvm/emulate.c +++ b/xen/arch/x86/hvm/emulate.c @@ -2918,7 +2918,7 @@ bool hvmemul_read_cache(const struct vcp unsigned int i; /* Cache unavailable? */ - if ( cache->num_ents > cache->max_ents ) + if ( !is_hvm_vcpu(v) || cache->num_ents > cache->max_ents ) return false; while ( size > sizeof(cache->ents->data) ) @@ -2950,7 +2950,7 @@ void hvmemul_write_cache(const struct vc unsigned int i; /* Cache unavailable? */ - if ( cache->num_ents > cache->max_ents ) + if ( !is_hvm_vcpu(v) || cache->num_ents > cache->max_ents ) return; while ( size > sizeof(cache->ents->data) ) --- a/xen/arch/x86/mm/guest_walk.c +++ b/xen/arch/x86/mm/guest_walk.c @@ -31,6 +31,7 @@ asm(".file \"" __OBJECT_FILE__ "\""); #include #include #include +#include /* * Modify a guest pagetable entry to set the Accessed and Dirty bits. @@ -80,9 +81,9 @@ static bool set_ad_bits(guest_intpte_t * * requested walk, to see whether the access is permitted. */ bool -guest_walk_tables(struct vcpu *v, struct p2m_domain *p2m, - unsigned long va, walk_t *gw, - uint32_t walk, mfn_t top_mfn, void *top_map) +guest_walk_tables(const struct vcpu *v, struct p2m_domain *p2m, + unsigned long va, walk_t *gw, uint32_t walk, + gfn_t top_gfn, mfn_t top_mfn, void *top_map) { struct domain *d = v->domain; p2m_type_t p2mt; @@ -91,8 +92,13 @@ guest_walk_tables(struct vcpu *v, struct #if GUEST_PAGING_LEVELS >= 4 /* 64-bit only... */ guest_l3e_t *l3p = NULL; guest_l4e_t *l4p; + paddr_t l4gpa; +#endif +#if GUEST_PAGING_LEVELS >= 3 /* PAE or 64... */ + paddr_t l3gpa; #endif uint32_t gflags, rc; + paddr_t l1gpa = 0, l2gpa = 0; unsigned int leaf_level; p2m_query_t qt = P2M_ALLOC | P2M_UNSHARE; @@ -133,7 +139,13 @@ guest_walk_tables(struct vcpu *v, struct /* Get the l4e from the top level table and check its flags*/ gw->l4mfn = top_mfn; l4p = (guest_l4e_t *) top_map; - gw->l4e = l4p[guest_l4_table_offset(va)]; + l4gpa = gfn_to_gaddr(top_gfn) + + guest_l4_table_offset(va) * sizeof(gw->l4e); + if ( !hvmemul_read_cache(v, l4gpa, &gw->l4e, sizeof(gw->l4e)) ) + { + gw->l4e = l4p[guest_l4_table_offset(va)]; + hvmemul_write_cache(v, l4gpa, &gw->l4e, sizeof(gw->l4e)); + } gflags = guest_l4e_get_flags(gw->l4e); if ( !(gflags & _PAGE_PRESENT) ) goto out; @@ -163,7 +175,13 @@ guest_walk_tables(struct vcpu *v, struct } /* Get the l3e and check its flags*/ - gw->l3e = l3p[guest_l3_table_offset(va)]; + l3gpa = gfn_to_gaddr(guest_l4e_get_gfn(gw->l4e)) + + guest_l3_table_offset(va) * sizeof(gw->l3e); + if ( !hvmemul_read_cache(v, l3gpa, &gw->l3e, sizeof(gw->l3e)) ) + { + gw->l3e = l3p[guest_l3_table_offset(va)]; + hvmemul_write_cache(v, l3gpa, &gw->l3e, sizeof(gw->l3e)); + } gflags = guest_l3e_get_flags(gw->l3e); if ( !(gflags & _PAGE_PRESENT) ) goto out; @@ -215,7 +233,14 @@ guest_walk_tables(struct vcpu *v, struct #else /* PAE only... */ /* Get the l3e and check its flag */ - gw->l3e = ((guest_l3e_t *) top_map)[guest_l3_table_offset(va)]; + l3gpa = gfn_to_gaddr(top_gfn) + ((unsigned long)top_map & ~PAGE_MASK) + + guest_l3_table_offset(va) * sizeof(gw->l3e); + if ( !hvmemul_read_cache(v, l3gpa, &gw->l3e, sizeof(gw->l3e)) ) + { + gw->l3e = ((guest_l3e_t *)top_map)[guest_l3_table_offset(va)]; + hvmemul_write_cache(v, l3gpa, &gw->l3e, sizeof(gw->l3e)); + } + gflags = guest_l3e_get_flags(gw->l3e); if ( !(gflags & _PAGE_PRESENT) ) goto out; @@ -241,18 +266,24 @@ guest_walk_tables(struct vcpu *v, struct goto out; } - /* Get the l2e */ - gw->l2e = l2p[guest_l2_table_offset(va)]; + l2gpa = gfn_to_gaddr(guest_l3e_get_gfn(gw->l3e)); #else /* 32-bit only... */ - /* Get l2e from the top level table */ gw->l2mfn = top_mfn; l2p = (guest_l2e_t *) top_map; - gw->l2e = l2p[guest_l2_table_offset(va)]; + l2gpa = gfn_to_gaddr(top_gfn); #endif /* All levels... */ + /* Get the l2e */ + l2gpa += guest_l2_table_offset(va) * sizeof(gw->l2e); + if ( !hvmemul_read_cache(v, l2gpa, &gw->l2e, sizeof(gw->l2e)) ) + { + gw->l2e = l2p[guest_l2_table_offset(va)]; + hvmemul_write_cache(v, l2gpa, &gw->l2e, sizeof(gw->l2e)); + } + /* Check the l2e flags. */ gflags = guest_l2e_get_flags(gw->l2e); if ( !(gflags & _PAGE_PRESENT) ) @@ -334,7 +365,15 @@ guest_walk_tables(struct vcpu *v, struct gw->pfec |= rc & PFEC_synth_mask; goto out; } - gw->l1e = l1p[guest_l1_table_offset(va)]; + + l1gpa = gfn_to_gaddr(guest_l2e_get_gfn(gw->l2e)) + + guest_l1_table_offset(va) * sizeof(gw->l1e); + if ( !hvmemul_read_cache(v, l1gpa, &gw->l1e, sizeof(gw->l1e)) ) + { + gw->l1e = l1p[guest_l1_table_offset(va)]; + hvmemul_write_cache(v, l1gpa, &gw->l1e, sizeof(gw->l1e)); + } + gflags = guest_l1e_get_flags(gw->l1e); if ( !(gflags & _PAGE_PRESENT) ) goto out; @@ -445,22 +484,34 @@ guest_walk_tables(struct vcpu *v, struct case 1: if ( set_ad_bits(&l1p[guest_l1_table_offset(va)].l1, &gw->l1e.l1, (walk & PFEC_write_access)) ) + { paging_mark_dirty(d, gw->l1mfn); + hvmemul_write_cache(v, l1gpa, &gw->l1e, sizeof(gw->l1e)); + } /* Fallthrough */ case 2: if ( set_ad_bits(&l2p[guest_l2_table_offset(va)].l2, &gw->l2e.l2, (walk & PFEC_write_access) && leaf_level == 2) ) + { paging_mark_dirty(d, gw->l2mfn); + hvmemul_write_cache(v, l2gpa, &gw->l2e, sizeof(gw->l2e)); + } /* Fallthrough */ #if GUEST_PAGING_LEVELS == 4 /* 64-bit only... */ case 3: if ( set_ad_bits(&l3p[guest_l3_table_offset(va)].l3, &gw->l3e.l3, (walk & PFEC_write_access) && leaf_level == 3) ) + { paging_mark_dirty(d, gw->l3mfn); + hvmemul_write_cache(v, l3gpa, &gw->l3e, sizeof(gw->l3e)); + } if ( set_ad_bits(&l4p[guest_l4_table_offset(va)].l4, &gw->l4e.l4, false) ) + { paging_mark_dirty(d, gw->l4mfn); + hvmemul_write_cache(v, l4gpa, &gw->l4e, sizeof(gw->l4e)); + } #endif } --- a/xen/arch/x86/mm/hap/guest_walk.c +++ b/xen/arch/x86/mm/hap/guest_walk.c @@ -91,7 +91,8 @@ unsigned long hap_p2m_ga_to_gfn(GUEST_PA #if GUEST_PAGING_LEVELS == 3 top_map += (cr3 & ~(PAGE_MASK | 31)); #endif - walk_ok = guest_walk_tables(v, p2m, ga, &gw, *pfec, top_mfn, top_map); + walk_ok = guest_walk_tables(v, p2m, ga, &gw, *pfec, + top_gfn, top_mfn, top_map); unmap_domain_page(top_map); put_page(top_page); --- a/xen/arch/x86/mm/shadow/multi.c +++ b/xen/arch/x86/mm/shadow/multi.c @@ -175,9 +175,13 @@ static inline bool sh_walk_guest_tables(struct vcpu *v, unsigned long va, walk_t *gw, uint32_t pfec) { + gfn_t root_gfn = _gfn(paging_mode_external(v->domain) + ? cr3_pa(v->arch.hvm.guest_cr[3]) >> PAGE_SHIFT + : pagetable_get_pfn(v->arch.guest_table)); + #if GUEST_PAGING_LEVELS == 3 /* PAE */ return guest_walk_tables(v, p2m_get_hostp2m(v->domain), va, gw, pfec, - INVALID_MFN, v->arch.paging.shadow.gl3e); + root_gfn, INVALID_MFN, v->arch.paging.shadow.gl3e); #else /* 32 or 64 */ const struct domain *d = v->domain; mfn_t root_mfn = (v->arch.flags & TF_kernel_mode @@ -185,7 +189,7 @@ sh_walk_guest_tables(struct vcpu *v, uns : pagetable_get_mfn(v->arch.guest_table_user)); void *root_map = map_domain_page(root_mfn); bool ok = guest_walk_tables(v, p2m_get_hostp2m(d), va, gw, pfec, - root_mfn, root_map); + root_gfn, root_mfn, root_map); unmap_domain_page(root_map); --- a/xen/include/asm-x86/guest_pt.h +++ b/xen/include/asm-x86/guest_pt.h @@ -428,8 +428,9 @@ static inline unsigned int guest_walk_to #define guest_walk_tables GPT_RENAME(guest_walk_tables, GUEST_PAGING_LEVELS) bool -guest_walk_tables(struct vcpu *v, struct p2m_domain *p2m, unsigned long va, - walk_t *gw, uint32_t pfec, mfn_t top_mfn, void *top_map); +guest_walk_tables(const struct vcpu *v, struct p2m_domain *p2m, + unsigned long va, walk_t *gw, uint32_t pfec, + gfn_t top_gfn, mfn_t top_mfn, void *top_map); /* Pretty-print the contents of a guest-walk */ static inline void print_gw(const walk_t *gw) From patchwork Fri Jan 31 16:45:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11360283 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B318F13A4 for ; Fri, 31 Jan 2020 16:46:37 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 98EB020663 for ; Fri, 31 Jan 2020 16:46:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 98EB020663 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZQM-0000IM-ES; Fri, 31 Jan 2020 16:45:42 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZQL-0000IA-E1 for xen-devel@lists.xenproject.org; Fri, 31 Jan 2020 16:45:41 +0000 X-Inumbo-ID: 1d49bbbe-4449-11ea-8bec-12813bfff9fa Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 1d49bbbe-4449-11ea-8bec-12813bfff9fa; Fri, 31 Jan 2020 16:45:40 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 0B62AAD3C; Fri, 31 Jan 2020 16:45:40 +0000 (UTC) From: Jan Beulich To: "xen-devel@lists.xenproject.org" References: Message-ID: <060e64dd-d107-1d1c-e2eb-da21559f8f5f@suse.com> Date: Fri, 31 Jan 2020 17:45:42 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.4.2 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Subject: [Xen-devel] [PATCH v4 6/7] x86/mm: drop p2mt parameter from map_domain_gfn() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: George Dunlap , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Paul Durrant Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" No caller actually consumes it. Signed-off-by: Jan Beulich Acked-by: Andrew Cooper --- v4: New. --- a/xen/arch/x86/hvm/dom0_build.c +++ b/xen/arch/x86/hvm/dom0_build.c @@ -278,7 +278,6 @@ static int __init pvh_add_mem_range(stru static int __init pvh_setup_vmx_realmode_helpers(struct domain *d) { - p2m_type_t p2mt; uint32_t rc, *ident_pt; mfn_t mfn; paddr_t gaddr; @@ -317,7 +316,7 @@ static int __init pvh_setup_vmx_realmode * superpages. */ ident_pt = map_domain_gfn(p2m_get_hostp2m(d), _gfn(PFN_DOWN(gaddr)), - &mfn, &p2mt, 0, &rc); + &mfn, 0, &rc); if ( ident_pt == NULL ) { printk("Unable to map identity page tables\n"); --- a/xen/arch/x86/mm/guest_walk.c +++ b/xen/arch/x86/mm/guest_walk.c @@ -86,7 +86,6 @@ guest_walk_tables(const struct vcpu *v, gfn_t top_gfn, mfn_t top_mfn, void *top_map) { struct domain *d = v->domain; - p2m_type_t p2mt; guest_l1e_t *l1p = NULL; guest_l2e_t *l2p = NULL; #if GUEST_PAGING_LEVELS >= 4 /* 64-bit only... */ @@ -165,7 +164,6 @@ guest_walk_tables(const struct vcpu *v, l3p = map_domain_gfn(p2m, guest_l4e_get_gfn(gw->l4e), &gw->l3mfn, - &p2mt, qt, &rc); if ( l3p == NULL ) @@ -257,7 +255,6 @@ guest_walk_tables(const struct vcpu *v, l2p = map_domain_gfn(p2m, guest_l3e_get_gfn(gw->l3e), &gw->l2mfn, - &p2mt, qt, &rc); if ( l2p == NULL ) @@ -357,7 +354,6 @@ guest_walk_tables(const struct vcpu *v, l1p = map_domain_gfn(p2m, guest_l2e_get_gfn(gw->l2e), &gw->l1mfn, - &p2mt, qt, &rc); if ( l1p == NULL ) --- a/xen/arch/x86/mm/hap/nested_ept.c +++ b/xen/arch/x86/mm/hap/nested_ept.c @@ -151,7 +151,6 @@ static uint32_t nept_walk_tables(struct vcpu *v, unsigned long l2ga, ept_walk_t *gw) { int lvl; - p2m_type_t p2mt; uint32_t rc = 0, ret = 0, gflags; struct domain *d = v->domain; struct p2m_domain *p2m = d->arch.p2m; @@ -163,7 +162,7 @@ nept_walk_tables(struct vcpu *v, unsigne for (lvl = 4; lvl > 0; lvl--) { - lxp = map_domain_gfn(p2m, base_gfn, &lxmfn, &p2mt, P2M_ALLOC, &rc); + lxp = map_domain_gfn(p2m, base_gfn, &lxmfn, P2M_ALLOC, &rc); if ( !lxp ) goto map_err; gw->lxe[lvl] = lxp[ept_lvl_table_offset(l2ga, lvl)]; --- a/xen/arch/x86/mm/p2m.c +++ b/xen/arch/x86/mm/p2m.c @@ -2214,8 +2214,9 @@ unsigned long paging_gva_to_gfn(struct v * synthetic/structure PFEC_* bits. */ void *map_domain_gfn(struct p2m_domain *p2m, gfn_t gfn, mfn_t *mfn, - p2m_type_t *p2mt, p2m_query_t q, uint32_t *pfec) + p2m_query_t q, uint32_t *pfec) { + p2m_type_t p2mt; struct page_info *page; if ( !gfn_valid(p2m->domain, gfn) ) @@ -2225,8 +2226,8 @@ void *map_domain_gfn(struct p2m_domain * } /* Translate the gfn, unsharing if shared. */ - page = p2m_get_page_from_gfn(p2m, gfn, p2mt, NULL, q); - if ( p2m_is_paging(*p2mt) ) + page = p2m_get_page_from_gfn(p2m, gfn, &p2mt, NULL, q); + if ( p2m_is_paging(p2mt) ) { ASSERT(p2m_is_hostp2m(p2m)); if ( page ) @@ -2235,7 +2236,7 @@ void *map_domain_gfn(struct p2m_domain * *pfec = PFEC_page_paged; return NULL; } - if ( p2m_is_shared(*p2mt) ) + if ( p2m_is_shared(p2mt) ) { if ( page ) put_page(page); --- a/xen/include/asm-x86/p2m.h +++ b/xen/include/asm-x86/p2m.h @@ -762,7 +762,7 @@ int __must_check p2m_set_entry(struct p2 extern void p2m_pt_init(struct p2m_domain *p2m); void *map_domain_gfn(struct p2m_domain *p2m, gfn_t gfn, mfn_t *mfn, - p2m_type_t *p2mt, p2m_query_t q, uint32_t *pfec); + p2m_query_t q, uint32_t *pfec); /* Debugging and auditing of the P2M code? */ #ifndef NDEBUG From patchwork Fri Jan 31 16:46:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11360285 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7FE9613A4 for ; Fri, 31 Jan 2020 16:46:50 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 664EB20663 for ; Fri, 31 Jan 2020 16:46:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 664EB20663 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZQj-0000Ng-Pn; Fri, 31 Jan 2020 16:46:05 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ixZQi-0000NV-SR for xen-devel@lists.xenproject.org; Fri, 31 Jan 2020 16:46:04 +0000 X-Inumbo-ID: 2b50b370-4449-11ea-8396-bc764e2007e4 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 2b50b370-4449-11ea-8396-bc764e2007e4; Fri, 31 Jan 2020 16:46:04 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 44324AC8F; Fri, 31 Jan 2020 16:46:03 +0000 (UTC) From: Jan Beulich To: "xen-devel@lists.xenproject.org" References: Message-ID: Date: Fri, 31 Jan 2020 17:46:06 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.4.2 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Subject: [Xen-devel] [PATCH v4 7/7] x86/HVM: reduce scope of pfec in hvm_emulate_init_per_insn() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: George Dunlap , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Paul Durrant Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" It needs calculating only in one out of three cases. Re-structure the code a little such that the variable truly gets calculated only when we don't get any insn bytes from elsewhere, and hence need to (try to) fetch them. Also OR in PFEC_insn_fetch right in the initializer. While in this mood, restrict addr's scope as well. Signed-off-by: Jan Beulich Acked-by: Andrew Cooper Reviewed-by: Paul Durrant --- v4: New. --- a/xen/arch/x86/hvm/emulate.c +++ b/xen/arch/x86/hvm/emulate.c @@ -2762,8 +2762,6 @@ void hvm_emulate_init_per_insn( unsigned int insn_bytes) { struct vcpu *curr = current; - unsigned int pfec = PFEC_page_present; - unsigned long addr; hvmemul_ctxt->ctxt.lma = hvm_long_mode_active(curr); @@ -2778,14 +2776,23 @@ void hvm_emulate_init_per_insn( hvmemul_ctxt->seg_reg[x86_seg_ss].db ? 32 : 16; } - if ( hvmemul_ctxt->seg_reg[x86_seg_ss].dpl == 3 ) - pfec |= PFEC_user_mode; - hvmemul_ctxt->insn_buf_eip = hvmemul_ctxt->ctxt.regs->rip; - if ( !insn_bytes ) + + if ( insn_bytes ) { + hvmemul_ctxt->insn_buf_bytes = insn_bytes; + memcpy(hvmemul_ctxt->insn_buf, insn_buf, insn_bytes); + } + else if ( !(hvmemul_ctxt->insn_buf_bytes = + hvm_get_insn_bytes(curr, hvmemul_ctxt->insn_buf)) ) + { + unsigned int pfec = PFEC_page_present | PFEC_insn_fetch; + unsigned long addr; + + if ( hvmemul_ctxt->seg_reg[x86_seg_ss].dpl == 3 ) + pfec |= PFEC_user_mode; + hvmemul_ctxt->insn_buf_bytes = - hvm_get_insn_bytes(curr, hvmemul_ctxt->insn_buf) ?: (hvm_virtual_to_linear_addr(x86_seg_cs, &hvmemul_ctxt->seg_reg[x86_seg_cs], hvmemul_ctxt->insn_buf_eip, @@ -2795,15 +2802,9 @@ void hvm_emulate_init_per_insn( &addr) && hvm_copy_from_guest_linear(hvmemul_ctxt->insn_buf, addr, sizeof(hvmemul_ctxt->insn_buf), - pfec | PFEC_insn_fetch, - NULL) == HVMTRANS_okay) ? + pfec, NULL) == HVMTRANS_okay) ? sizeof(hvmemul_ctxt->insn_buf) : 0; } - else - { - hvmemul_ctxt->insn_buf_bytes = insn_bytes; - memcpy(hvmemul_ctxt->insn_buf, insn_buf, insn_bytes); - } } void hvm_emulate_writeback(