From patchwork Fri Feb 20 14:29:05 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 5856411 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 19A409F269 for ; Fri, 20 Feb 2015 14:32:14 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 395BC203F7 for ; Fri, 20 Feb 2015 14:32:13 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5AE9820364 for ; Fri, 20 Feb 2015 14:32:12 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1YOoaL-0001ew-Sm; Fri, 20 Feb 2015 14:29:41 +0000 Received: from mx1.redhat.com ([209.132.183.28]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1YOoaJ-0001bG-Ix for linux-arm-kernel@lists.infradead.org; Fri, 20 Feb 2015 14:29:40 +0000 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t1KETBdV008059 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 20 Feb 2015 09:29:11 -0500 Received: from hawk.usersys.redhat.com ([10.34.1.145]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t1KET6NJ024242 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO); Fri, 20 Feb 2015 09:29:08 -0500 Date: Fri, 20 Feb 2015 15:29:05 +0100 From: Andrew Jones To: Paolo Bonzini Subject: Re: [RFC/RFT PATCH 0/3] arm64: KVM: work around incoherency with uncached guest mappings Message-ID: <20150220142905.GA10942@hawk.usersys.redhat.com> References: <1424343286-6792-1-git-send-email-ard.biesheuvel@linaro.org> <20150219165731.GA26965@hawk.usersys.redhat.com> <20150219175518.GB26965@hawk.usersys.redhat.com> <54E62404.9040702@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <54E62404.9040702@redhat.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20150220_062939_701135_2DA53822 X-CRM114-Status: GOOD ( 18.47 ) X-Spam-Score: -6.0 (------) Cc: Peter Maydell , KVM devel mailing list , Ard Biesheuvel , Marc Zyngier , Christoffer Dall , Laszlo Ersek , "kvmarm@lists.cs.columbia.edu" , "linux-arm-kernel@lists.infradead.org" X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-5.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Thu, Feb 19, 2015 at 06:57:24PM +0100, Paolo Bonzini wrote: > > > On 19/02/2015 18:55, Andrew Jones wrote: > >> > > (I don't have an exact number for how many times it went to EL1 because > >> > > access_mair() doesn't have a trace point.) > >> > > (I got the 62873 number by testing a 3rd kernel build that only had patch > >> > > 3/3 applied to the base, and counting kvm_toggle_cache events.) > >> > > (The number 50 is the number of kvm_toggle_cache events *without* 3/3 > >> > > applied.) > >> > > > >> > > I consider this bad news because, even considering it only goes to EL2, > >> > > it goes a ton more than it used to. I realize patch 3/3 isn't the final > >> > > plan for enabling traps though. > > If a full guest boots, can you try timing a kernel compile? > Guests boot. I used an 8 vcpu, 14G memory guest; compiled the kernel 4 times inside the guest for each host kernel; base and mair. I dropped the time from the first run of each set, and captured the other 3. Command line used below. Time is from the Elapsed (wall clock) time (h:mm:ss or m:ss): output of /usr/bin/time - the host's wall clock. /usr/bin/time --verbose ssh $VM 'cd kernel && make -s clean && make -s -j8' Results: base: 3:06.11 3:07.00 3:10.93 mair: 3:08.47 3:06.75 3:04.76 So looks like the 3 orders of magnitude greater number of traps (only to el2) don't impact kernel compiles. Then I thought I'd be able to quick measure the number of cycles a trap to el2 takes with this kvm-unit-tests test int main(void) { unsigned long start, end; unsigned int sctlr; asm volatile( " mrs %0, sctlr_el1\n" " msr pmcr_el0, %1\n" : "=&r" (sctlr) : "r" (5)); asm volatile( " mrs %0, pmccntr_el0\n" " msr sctlr_el1, %2\n" " mrs %1, pmccntr_el0\n" : "=&r" (start), "=&r" (end) : "r" (sctlr)); printf("%llx\n", end - start); return 0; } after applying this patch to kvm But I get zero for the cycle count. Not sure what I'm missing. drew diff --git a/arch/arm64/kvm/hyp.S b/arch/arm64/kvm/hyp.S index bb91b6fc63861..5de39d740aa58 100644 --- a/arch/arm64/kvm/hyp.S +++ b/arch/arm64/kvm/hyp.S @@ -770,7 +770,7 @@ mrs x2, mdcr_el2 and x2, x2, #MDCR_EL2_HPMN_MASK - orr x2, x2, #(MDCR_EL2_TPM | MDCR_EL2_TPMCR) +// orr x2, x2, #(MDCR_EL2_TPM | MDCR_EL2_TPMCR) orr x2, x2, #(MDCR_EL2_TDRA | MDCR_EL2_TDOSA) // Check for KVM_ARM64_DEBUG_DIRTY, and set debug to trap