From patchwork Mon Jun 18 13:19:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Cao, Lei" X-Patchwork-Id: 10471421 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 713D56032A for ; Mon, 18 Jun 2018 13:26:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 617D32837D for ; Mon, 18 Jun 2018 13:26:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 55EFD289CB; Mon, 18 Jun 2018 13:26:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 72C8E2837D for ; Mon, 18 Jun 2018 13:26:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934189AbeFRN0G (ORCPT ); Mon, 18 Jun 2018 09:26:06 -0400 Received: from us-smtp-delivery-131.mimecast.com ([63.128.21.131]:34962 "EHLO us-smtp-delivery-131.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933517AbeFRN0F (ORCPT ); Mon, 18 Jun 2018 09:26:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=StratusTechnologies.onmicrosoft.com; s=selector1-stratus-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DMm2Fad48xDqVol6HeWRnn66QUEq+Yo3tZgPaMiD2Yk=; b=A0jz+D/tD2qrLqhy2+y2z9/LVq1bwDogZyGV/de0oEomynfdhFY4uHyCuATBphRJdWXzv6LDUC8AoV3dKy7c4apqIG7ga9wpIqDJlTx7fhb3xTwheIQt3cxkMG6JfGT7W24JZUZ5O05xMUhBX88IalHwAHdswHZRl2g2E5XtJGs= Received: from NAM01-BY2-obe.outbound.protection.outlook.com (mail-by2nam01lp0182.outbound.protection.outlook.com [216.32.181.182]) (Using TLS) by us-smtp-1.mimecast.com with ESMTP id us-mta-168-VNTZ-y4PM5W356-s8hEahg-1; Mon, 18 Jun 2018 09:19:57 -0400 Received: from BYAPR08MB3973.namprd08.prod.outlook.com (52.135.194.143) by BYAPR08MB3864.namprd08.prod.outlook.com (52.135.193.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.863.19; Mon, 18 Jun 2018 13:19:55 +0000 Received: from BYAPR08MB3973.namprd08.prod.outlook.com ([fe80::a8c6:d203:dc46:b5ec]) by BYAPR08MB3973.namprd08.prod.outlook.com ([fe80::a8c6:d203:dc46:b5ec%3]) with mapi id 15.20.0863.016; Mon, 18 Jun 2018 13:19:55 +0000 From: "Cao, Lei" To: "kvm@vger.kernel.org" Subject: [PATCH v3 2/6] KVM: move running VCPU from ARM to common code Thread-Topic: [PATCH v3 2/6] KVM: move running VCPU from ARM to common code Thread-Index: AQHUBwcM+jwIh1WHu0OxG0w/rE5gkg== Date: Mon, 18 Jun 2018 13:19:55 +0000 Message-ID: References: <201806181256.w5ICuJdC024388@dev1.evr.stratus.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [198.97.42.5] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; BYAPR08MB3864; 7:E7K0V5csq1hzduGGDOpgpZWU5h6Swa41RS6R/4cOnXeAMDx22wbtZ74CIa2EQ9WDZDHkA0uewHlFelIbrJC3+YGGhg69fDALoLc7zA21S4ZmgapGCyoy3NBqSD8JkJAc0LKR62zfBnHrWYzaz5iQZhqAeApTTECsG0auHALwB6antk6yMj+kJfD33ppMehgkkD95eeWg7zfgMkQdcBZK8CI9UrWHDD8xC1uyismujaTO8VguccBhoCABMx59YVtx; 20:50u6bsYZqYr7jG5sp1hV/p0lAlNZF3zfdgDzvu0R2Cs2h6s0fl8JEiktnyHQMqX+9nI8ybkEs/mNOjPaJHUqwGRC3fjBhxOmWapWGJcDUE/8KkrD7C67ilzPzFv3JIRpppruSOKi5QLsHRiUwYFo2u+QjI4ZpPN7fU6dmQOVJlQ= x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: b25180d8-56ad-48e1-8a47-08d5d51e2f0f x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(4534165)(4627221)(201703031133081)(201702281549075)(5600026)(711020)(2017052603328)(7153060)(7193020); SRVR:BYAPR08MB3864; x-ms-traffictypediagnostic: BYAPR08MB3864: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(93001095)(3231254)(944501410)(52105095)(149027)(150027)(6041310)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123562045)(20161123558120)(6072148)(201708071742011)(7699016); SRVR:BYAPR08MB3864; BCL:0; PCL:0; RULEID:; SRVR:BYAPR08MB3864; x-forefront-prvs: 0707248B64 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(396003)(39380400002)(376002)(39850400004)(346002)(366004)(189003)(199004)(446003)(97736004)(476003)(316002)(86362001)(486006)(575784001)(2906002)(305945005)(7736002)(2501003)(5660300001)(3660700001)(3280700002)(66066001)(2351001)(25786009)(6116002)(14454004)(3846002)(72206003)(478600001)(55016002)(53936002)(102836004)(26005)(7696005)(186003)(8676002)(6506007)(99286004)(5640700003)(2900100001)(5250100002)(106356001)(68736007)(105586002)(6916009)(8936002)(33656002)(81166006)(59450400001)(81156014)(76176011)(1730700003)(9686003)(74316002)(6436002); DIR:OUT; SFP:1102; SCL:1; SRVR:BYAPR08MB3864; H:BYAPR08MB3973.namprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; x-microsoft-antispam-message-info: V2s/uDA7SaWLyOZEkplJZ1Hr2o1NMzgZ0j7JK/73ri/2AeRF7PJCRWXkHjfpyy+BFpzqVcMfhpRCtIjkKB+Bmwb9Q1yMz657BGDxnP5WeBSHTiESOhlaWi1fvcPXSg+PQAXQ0dNKTgbWMqaaDR7oynOdzxCD+yw+I1AqR2MxUMI56LLwsn3AXC1IPW9xGgo8 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: stratus.com X-MS-Exchange-CrossTenant-Network-Message-Id: b25180d8-56ad-48e1-8a47-08d5d51e2f0f X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Jun 2018 13:19:55.6982 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: de36b473-b8ad-46ff-837f-9da16b8d1b77 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR08MB3864 X-MC-Unique: VNTZ-y4PM5W356-s8hEahg-1 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For ring-based dirty log tracking, it will be more efficient to account writes during schedule-out or schedule-in to the currently running VCPU. We would like to do it even if the write doesn't use the current VCPU's address space, as is the case for cached writes (see commit 4e335d9e7ddb, "Revert "KVM: Support vCPU-based gfn->hva cache"", 2017-05-02). Therefore, add a mechanism to track the currently-loaded kvm_vcpu struct. There is already something similar in KVM/ARM; one important difference is that kvm_arch_vcpu_{load,put} have two callers in virt/kvm/kvm_main.c: we have to update both the architecture-independent vcpu_{load,put} and the preempt notifiers. Another change made in the process is to allow using kvm_get_running_vcpu() in preemptible code. This is allowed because preempt notifiers ensure that the value does not change even after the VCPU thread is migrated. Signed-off-by: Paolo Bonzini --- arch/arm/include/asm/kvm_host.h | 2 -- arch/arm64/include/asm/kvm_host.h | 2 -- include/linux/kvm_host.h | 3 +++ virt/kvm/arm/arm.c | 30 ------------------------------ virt/kvm/arm/perf.c | 6 +++--- virt/kvm/arm/vgic/vgic-init.c | 2 +- virt/kvm/arm/vgic/vgic-mmio.c | 2 +- virt/kvm/kvm_main.c | 25 ++++++++++++++++++++++++- 8 files changed, 32 insertions(+), 40 deletions(-) diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h index a9f7d3f..c3a260a 100644 --- a/arch/arm/include/asm/kvm_host.h +++ b/arch/arm/include/asm/kvm_host.h @@ -225,8 +225,6 @@ int kvm_unmap_hva_range(struct kvm *kvm, int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end); int kvm_test_age_hva(struct kvm *kvm, unsigned long hva); -struct kvm_vcpu *kvm_arm_get_running_vcpu(void); -struct kvm_vcpu __percpu **kvm_get_running_vcpus(void); void kvm_arm_halt_guest(struct kvm *kvm); void kvm_arm_resume_guest(struct kvm *kvm); diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index ea6cb5b..e8571be 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -327,8 +327,6 @@ int kvm_unmap_hva_range(struct kvm *kvm, int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end); int kvm_test_age_hva(struct kvm *kvm, unsigned long hva); -struct kvm_vcpu *kvm_arm_get_running_vcpu(void); -struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void); void kvm_arm_halt_guest(struct kvm *kvm); void kvm_arm_resume_guest(struct kvm *kvm); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index cd2f52d..11e891a 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1236,6 +1236,9 @@ static inline void kvm_vcpu_set_dy_eligible(struct kvm_vcpu *vcpu, bool val) } #endif /* CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT */ +struct kvm_vcpu *kvm_get_running_vcpu(void); +struct kvm_vcpu __percpu **kvm_get_running_vcpus(void); + #ifdef CONFIG_HAVE_KVM_IRQ_BYPASS bool kvm_arch_has_irq_bypass(void); int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *, diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c index cd7d90c..24d186a 100644 --- a/virt/kvm/arm/arm.c +++ b/virt/kvm/arm/arm.c @@ -56,9 +56,6 @@ static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); static kvm_cpu_context_t __percpu *kvm_host_cpu_state; -/* Per-CPU variable containing the currently running vcpu. */ -static DEFINE_PER_CPU(struct kvm_vcpu *, kvm_arm_running_vcpu); - /* The VMID used in the VTTBR */ static atomic64_t kvm_vmid_gen = ATOMIC64_INIT(1); static u32 kvm_next_vmid; @@ -69,30 +66,6 @@ static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled); -static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu) -{ - BUG_ON(preemptible()); - __this_cpu_write(kvm_arm_running_vcpu, vcpu); -} - -/** - * kvm_arm_get_running_vcpu - get the vcpu running on the current CPU. - * Must be called from non-preemptible context - */ -struct kvm_vcpu *kvm_arm_get_running_vcpu(void) -{ - BUG_ON(preemptible()); - return __this_cpu_read(kvm_arm_running_vcpu); -} - -/** - * kvm_arm_get_running_vcpus - get the per-CPU array of currently running vcpus. - */ -struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void) -{ - return &kvm_arm_running_vcpu; -} - int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE; @@ -356,7 +329,6 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) vcpu->cpu = cpu; vcpu->arch.host_cpu_context = this_cpu_ptr(kvm_host_cpu_state); - kvm_arm_set_running_vcpu(vcpu); kvm_vgic_load(vcpu); kvm_timer_vcpu_load(vcpu); } @@ -367,8 +339,6 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) kvm_vgic_put(vcpu); vcpu->cpu = -1; - - kvm_arm_set_running_vcpu(NULL); } static void vcpu_power_off(struct kvm_vcpu *vcpu) diff --git a/virt/kvm/arm/perf.c b/virt/kvm/arm/perf.c index 1a3849d..ca511df 100644 --- a/virt/kvm/arm/perf.c +++ b/virt/kvm/arm/perf.c @@ -24,14 +24,14 @@ static int kvm_is_in_guest(void) { - return kvm_arm_get_running_vcpu() != NULL; + return kvm_get_running_vcpu() != NULL; } static int kvm_is_user_mode(void) { struct kvm_vcpu *vcpu; - vcpu = kvm_arm_get_running_vcpu(); + vcpu = kvm_get_running_vcpu(); if (vcpu) return !vcpu_mode_priv(vcpu); @@ -43,7 +43,7 @@ static unsigned long kvm_get_guest_ip(void) { struct kvm_vcpu *vcpu; - vcpu = kvm_arm_get_running_vcpu(); + vcpu = kvm_get_running_vcpu(); if (vcpu) return *vcpu_pc(vcpu); diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c index 6231012..bd0f969 100644 --- a/virt/kvm/arm/vgic/vgic-init.c +++ b/virt/kvm/arm/vgic/vgic-init.c @@ -307,7 +307,7 @@ int vgic_init(struct kvm *kvm) * when we just initialized the VGIC. */ preempt_disable(); - vcpu = kvm_arm_get_running_vcpu(); + vcpu = kvm_get_running_vcpu(); if (vcpu) kvm_vgic_load(vcpu); preempt_enable(); diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c index deb51ee..c0f54a0 100644 --- a/virt/kvm/arm/vgic/vgic-mmio.c +++ b/virt/kvm/arm/vgic/vgic-mmio.c @@ -201,7 +201,7 @@ static void vgic_mmio_change_active(struct kvm_vcpu *vcpu, struct vgic_irq *irq, * when accessing VGIC state from user space so irq->vcpu->cpu is * always -1. */ - requester_vcpu = kvm_arm_get_running_vcpu(); + requester_vcpu = kvm_get_running_vcpu(); /* * If this virtual IRQ was written into a list register, we diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index f8af0d9..6e4d71c 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -104,6 +104,7 @@ EXPORT_SYMBOL_GPL(kvm_vcpu_cache); static __read_mostly struct preempt_ops kvm_preempt_ops; +static DEFINE_PER_CPU(struct kvm_vcpu *, kvm_running_vcpu); struct dentry *kvm_debugfs_dir; EXPORT_SYMBOL_GPL(kvm_debugfs_dir); @@ -154,6 +155,8 @@ bool kvm_is_reserved_pfn(kvm_pfn_t pfn) void vcpu_load(struct kvm_vcpu *vcpu) { int cpu = get_cpu(); + + __this_cpu_write(kvm_running_vcpu, vcpu); preempt_notifier_register(&vcpu->preempt_notifier); kvm_arch_vcpu_load(vcpu, cpu); put_cpu(); @@ -165,6 +168,7 @@ void vcpu_put(struct kvm_vcpu *vcpu) preempt_disable(); kvm_arch_vcpu_put(vcpu); preempt_notifier_unregister(&vcpu->preempt_notifier); + __this_cpu_write(kvm_running_vcpu, NULL); preempt_enable(); } EXPORT_SYMBOL_GPL(vcpu_put); @@ -3983,8 +3987,8 @@ static void kvm_sched_in(struct preempt_notifier *pn, int cpu) if (vcpu->preempted) vcpu->preempted = false; + __this_cpu_write(kvm_running_vcpu, vcpu); kvm_arch_sched_in(vcpu, cpu); - kvm_arch_vcpu_load(vcpu, cpu); } @@ -3996,6 +4000,25 @@ static void kvm_sched_out(struct preempt_notifier *pn, if (current->state == TASK_RUNNING) vcpu->preempted = true; kvm_arch_vcpu_put(vcpu); + __this_cpu_write(kvm_running_vcpu, NULL); +} + +/** + * kvm_get_running_vcpu - get the vcpu running on the current CPU. + * Thanks to preempt notifiers, this can also be called from + * preemptible context. + */ +struct kvm_vcpu *kvm_get_running_vcpu(void) +{ + return __this_cpu_read(kvm_running_vcpu); +} + +/** + * kvm_get_running_vcpus - get the per-CPU array of currently running vcpus. + */ +struct kvm_vcpu * __percpu *kvm_get_running_vcpus(void) +{ + return &kvm_running_vcpu; } int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,