From patchwork Fri Feb 3 12:33:20 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 9554047 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 81724602B7 for ; Fri, 3 Feb 2017 12:34:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6F09A2823D for ; Fri, 3 Feb 2017 12:34:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6061E2847D; Fri, 3 Feb 2017 12:34:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_SORBS_SPAM,T_DKIM_INVALID autolearn=no version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D48402823D for ; Fri, 3 Feb 2017 12:34:52 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1cZd4k-00007u-BS; Fri, 03 Feb 2017 12:34:50 +0000 Received: from mail-wm0-f53.google.com ([74.125.82.53]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1cZd4g-0008Os-RW for linux-arm-kernel@lists.infradead.org; Fri, 03 Feb 2017 12:34:49 +0000 Received: by mail-wm0-f53.google.com with SMTP id t18so14325583wmt.0 for ; Fri, 03 Feb 2017 04:34:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=ah2pOG9Io/KV943zki0aTMamHLtC5IGVe7OV1Xgj9f8=; b=iwrGYdYRxS5WewOAdPebizvuTk9mMH4KEuvjl42ojT/I1Kikb/ZO449YdJxmLeL63N cjEedx978UR/bN2CGAw4HkKOugNHqb519NsiPhXf+HmOlGKt6omsHu7v7vLCZN7aCeHA u9gRHdgfgiP+Y+1ZXf1oSTGhRJU83cD9CopHM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=ah2pOG9Io/KV943zki0aTMamHLtC5IGVe7OV1Xgj9f8=; b=HyBp4ksvJJqnMop+623ZPdjDQ4d1ZAal2t9h7Jl+WUwE3uvzDY55Th3o7CPkAuTI3Q xq02INxYbN2kXs9cp2vbIOOKMEp7/Krw7CcoeCVvwKDGwoO+68CYups7KLghao9H/63I XEIjx8IY7cDBmvu1xjdukpX7ocNGJkTxKLomXhyMddI5vBzo+ddta7r0vXzA3lquCzYF iM4vIW1PHNcIiZ7dfF11pcxHagsjM1ikWv+yYEjLxGuy2YaII3501uhVQdXUUZW+awLv K1mDpNCG6hT3KRFWlhzvS0Sf/FGQVYhpNS6PF3rdF1ikiowh16SeiBnXuZJ2aKln2NFF 57AA== X-Gm-Message-State: AMke39npNhxeHVVFkC2fEdSNYy5tOCdPNtvmvXWj8VoTRvNOHIttcqagSDVK0dUqcvgesGlS X-Received: by 10.28.98.194 with SMTP id w185mr1348634wmb.84.1486125201893; Fri, 03 Feb 2017 04:33:21 -0800 (PST) Received: from localhost ([217.61.220.40]) by smtp.gmail.com with ESMTPSA id a35sm45151940wra.21.2017.02.03.04.33.21 (version=TLS1_2 cipher=AES128-SHA bits=128/128); Fri, 03 Feb 2017 04:33:21 -0800 (PST) Date: Fri, 3 Feb 2017 13:33:20 +0100 From: Christoffer Dall To: Jintack Lim Subject: Re: [RFC v3 00/10] Provide the EL1 physical timer to the VM Message-ID: <20170203123320.GK27852@cbox> References: <1485970990-13775-1-git-send-email-jintack@cs.columbia.edu> <20170202123104.GJ27852@cbox> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170203_043447_054343_21C9CECB X-CRM114-Status: GOOD ( 33.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: KVM General , Radim =?utf-8?B?S3LEjW3DocWZ?= , Marc Zyngier , Catalin Marinas , Will Deacon , linux@armlinux.org.uk, lkml - Kernel Mailing List , arm-mail-list , Andre Przywara , Paolo Bonzini , kvmarm@lists.cs.columbia.edu, Christoffer Dall Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP On Thu, Feb 02, 2017 at 09:51:13AM -0500, Jintack Lim wrote: > On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall wrote: > > Hi Jintack, > > > > On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote: > >> The ARM architecture defines the EL1 physical timer and the virtual timer, > >> and it is reasonable for an OS to expect to be able to access both. > >> However, the current KVM implementation does not provide the EL1 physical > >> timer to VMs but terminates VMs on access to the timer. > >> > >> This patch series enables VMs to use the EL1 physical timer through > >> trap-and-emulate. The KVM host emulates each EL1 physical timer register > >> access and sets up the background timer accordingly. When the background > >> timer expires, the KVM host injects EL1 physical timer interrupts to the > >> VM. Alternatively, it's also possible to allow VMs to access the EL1 > >> physical timer without trapping. However, this requires somehow using the > >> EL2 physical timer for the Linux host while running the VM instead of the > >> EL1 physical timer. Right now I just implemented trap-and-emulate because > >> this was straightforward to do, and I leave it to future work to determine > >> if transferring the EL1 physical timer state to the EL2 timer provides any > >> performance benefit. > >> > >> This feature will be useful for any OS that wishes to access the EL1 > >> physical timer. Nested virtualization is one of those use cases. A nested > >> hypervisor running inside a VM would think it has full access to the > >> hardware and naturally tries to use the EL1 physical timer as Linux would > >> do. Other nested hypervisors may try to use the EL2 physical timer as Xen > >> would do, but supporting the EL2 physical timer to the VM is out of scope > >> of this patch series. This patch series will make it easy to add the EL2 > >> timer support in the future, though. > >> > >> Note that Linux VMs booting in EL1 will be unaffected by this patch series > >> and will continue to use only the virtual timer and this patch series will > >> therefore not introduce any performance degredation as a result of > >> trap-and-emulate. > >> > >> v2 => v3: > >> - Rebase on kvmarm/queue > >> - Take kvm->lock to synchronize cntvoff across all vtimers > >> - Remove unnecessary function parameters > >> - Add comments > > > > I just gave v3 a test run on my TC2 (32-bit platform) and my guest > > quickly locks up trying to run cyclictest or when booting the machine it > > stalls with RCU timeouts. > > Ok. It's my fault not to specify that the emulated physical timer is > supported/tested on arm64. > On 32-bit platform, it is supposed to show the same behavior as > before, but I haven't tested. > Were you using the physical timer or the virtual timer for the guest? > > > > > Could you have a look? > > Sure, I'll have a look. I don't have access to my Cubietruck today, > but I can work on that tomorrow. > Don't bother, I've figured this out for you. You need the following fixup to your patch: This is an amuzing one. Thanks, -Christoffer diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c index 93c811c..35d7100 100644 --- a/virt/kvm/arm/arch_timer.c +++ b/virt/kvm/arm/arch_timer.c @@ -410,14 +410,21 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu, } /* Make the updates of cntvoff for all vtimer contexts atomic */ -static void update_vtimer_cntvoff(struct kvm *kvm, u64 cntvoff) +static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff) { int i; - struct kvm_vcpu *vcpu; + struct kvm *kvm = vcpu->kvm; + struct kvm_vcpu *tmp; mutex_lock(&kvm->lock); - kvm_for_each_vcpu(i, vcpu, kvm) - vcpu_vtimer(vcpu)->cntvoff = cntvoff; + kvm_for_each_vcpu(i, tmp, kvm) + vcpu_vtimer(tmp)->cntvoff = cntvoff; + + /* + * When called from the vcpu create path, the CPU being created is not + * included in the loop above, so we just set it here as well. + */ + vcpu_vtimer(vcpu)->cntvoff = cntvoff; mutex_unlock(&kvm->lock); } @@ -426,7 +433,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu) struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu; /* Synchronize cntvoff across all vtimers of a VM. */ - update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read()); + update_vtimer_cntvoff(vcpu, kvm_phys_timer_read()); vcpu_ptimer(vcpu)->cntvoff = 0; INIT_WORK(&timer->expired, kvm_timer_inject_irq_work); @@ -448,7 +455,7 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value) vtimer->cnt_ctl = value; break; case KVM_REG_ARM_TIMER_CNT: - update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read() - value); + update_vtimer_cntvoff(vcpu, kvm_phys_timer_read() - value); break; case KVM_REG_ARM_TIMER_CVAL: vtimer->cnt_cval = value;