From patchwork Wed Feb 20 20:15:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822709 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C34641399 for ; Wed, 20 Feb 2019 20:22:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AFD422F59C for ; Wed, 20 Feb 2019 20:22:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9FC392F50D; Wed, 20 Feb 2019 20:22:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E87D2F40B for ; Wed, 20 Feb 2019 20:22:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726137AbfBTURV (ORCPT ); Wed, 20 Feb 2019 15:17:21 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:34840 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725798AbfBTURV (ORCPT ); Wed, 20 Feb 2019 15:17:21 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK9bEb088246; Wed, 20 Feb 2019 20:16:58 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=Ndvkg+/D0hf5OrbzmK4FZM7exvpdfllBJBc6SNXVjDs=; b=JlODJvHfDFO9wDU6O8Ik5ea3sjo8fA4wa2mCxi9Mtn/xMBfEjH619kIAAlqxOV1Pc1sc BYtrkjN8hMfhqJNcrclZevU6gg++bQ/a+BkFmKTdcvG6tFZ/RH6SMbvtZCd1wnwtCMW+ VUsbcxGSFRVZVY+USdj1rGh/AB7M5hS9so5stCJHMvmPoo2FzxuYiESZqcKD2bSA0hXq TysLp0BmdElEjX28+jnOPq1sH0hqX7ON2pfaKhfzdHFglMoAmvhb0Jgmt70Jac5N2ayK 0Yx4MXArOtTvfX5BfwUN8t70UfVEjnKoiPZwghXYXqj1XJzsIXt7fJTU8IPIAItqrgpl Vw== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2qpb5rktkw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:16:58 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKGvIO002290 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:16:57 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKGuos009112; Wed, 20 Feb 2019 20:16:56 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:16:55 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 01/39] KVM: x86: fix Xen hypercall page msr handling Date: Wed, 20 Feb 2019 20:15:31 +0000 Message-Id: <20190220201609.28290-2-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=973 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Xen usually places its MSR at 0x4000000 or 0x4000200 depending on whether it is running in viridian mode or not. Note that this is not ABI guaranteed, so it is possible for Xen to advertise the MSR some place else. Given the way xen_hvm_config() is handled, if the former address is selected, this will conflict with HyperV's MSR (HV_X64_MSR_GUEST_OS_ID) which uses the same address. Given that the MSR location is arbitrary, move the xen_hvm_config() handling to the top of kvm_set_msr_common() before falling through. Signed-off-by: Joao Martins Reviewed-by: David Woodhouse --- arch/x86/kvm/x86.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 65e4559eef2f..47360a4b0d42 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2429,6 +2429,9 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) u32 msr = msr_info->index; u64 data = msr_info->data; + if (msr && (msr == vcpu->kvm->arch.xen_hvm_config.msr)) + return xen_hvm_config(vcpu, data); + switch (msr) { case MSR_AMD64_NB_CFG: case MSR_IA32_UCODE_WRITE: @@ -2644,8 +2647,6 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) vcpu->arch.msr_misc_features_enables = data; break; default: - if (msr && (msr == vcpu->kvm->arch.xen_hvm_config.msr)) - return xen_hvm_config(vcpu, data); if (kvm_pmu_is_valid_msr(vcpu, msr)) return kvm_pmu_set_msr(vcpu, msr_info); if (!ignore_msrs) { From patchwork Wed Feb 20 20:15:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822707 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A1881922 for ; Wed, 20 Feb 2019 20:22:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8D30F2F41D for ; Wed, 20 Feb 2019 20:22:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7F3212F2F8; Wed, 20 Feb 2019 20:22:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 87DAB2F318 for ; Wed, 20 Feb 2019 20:22:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726648AbfBTURX (ORCPT ); Wed, 20 Feb 2019 15:17:23 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:51690 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726003AbfBTURW (ORCPT ); Wed, 20 Feb 2019 15:17:22 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8Ves081690; Wed, 20 Feb 2019 20:17:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=fXX8HPeYvgLyAsNwOKMQrzfQAx3bkpNv8PCYQeoxzNk=; b=sWlsxj0vtZn/NVJSZt01UcBLann83WVuDzngwTYG+BzZsyzq/zAKQNFxJSROxQl0HI8q XAdaYcFH/Lz7HPusG6LV3/wEPnULjR+zon1K5ox/HVAL8+yENIeIAnAKgnftLM6RY9uZ jF22FjM59HRXp5heoeSkNA+5CbGA6GREcwiTusqhej9fhr1JTk1BaeLi+WoeJVJBo29w IcuCy44HzgviEj2ImAdEQTP7olRXN0A4eUXZzwOc+l9CEO8Iq5UutnboFwqMtB7/dk83 OBPAwAXGeU0HUq/WRTF+ZcNM0vl2xlJa7g+3W2XeMQ/pL5GcTlEbGckZ01XVbDDk0ajm Iw== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2130.oracle.com with ESMTP id 2qp9xu3xdc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:00 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKH0dr024494 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:00 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKGx7U015198; Wed, 20 Feb 2019 20:16:59 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:16:58 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 02/39] KVM: x86/xen: intercept xen hypercalls if enabled Date: Wed, 20 Feb 2019 20:15:32 +0000 Message-Id: <20190220201609.28290-3-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=940 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add a new exit reason for emulator to handle Xen hypercalls. Albeit these are injected only if guest has initialized the Xen hypercall page - the hypercall is just a convenience but one that is done by pretty much all guests. Hence if the guest sets the hypercall page, we assume a Xen guest is going to be set up. Emulator will then panic with: KVM: unknown exit reason 28 RAX=0000000000000011 RBX=ffffffff81e03e94 RCX=0000000040000000 RDX=0000000000000000 RSI=ffffffff81e03e70 RDI=0000000000000006 RBP=ffffffff81e03e90 RSP=ffffffff81e03e68 R8 =73726576206e6558 R9 =ffffffff81e03e90 R10=ffffffff81e03e94 R11=2e362e34206e6f69 R12=0000000040000004 R13=ffffffff81e03e8c R14=ffffffff81e03e88 R15=0000000000000000 RIP=ffffffff81001228 RFL=00000082 [--S----] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 0000000000000000 ffffffff 00c00000 CS =0010 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA] SS =0000 0000000000000000 ffffffff 00c00000 DS =0000 0000000000000000 ffffffff 00c00000 FS =0000 0000000000000000 ffffffff 00c00000 GS =0000 ffffffff81f34000 ffffffff 00c00000 LDT=0000 0000000000000000 ffffffff 00c00000 TR =0020 0000000000000000 00000fff 00808b00 DPL=0 TSS64-busy GDT= ffffffff81f3c000 0000007f IDT= ffffffff83265000 00000fff CR0=80050033 CR2=ffff880001fa6ff8 CR3=0000000001fa6000 CR4=000406a0 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000d01 Code=cc cc cc cc cc cc cc cc cc cc cc cc b8 11 00 00 00 0f 01 c1 cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc b8 12 00 00 00 0f Signed-off-by: Joao Martins Signed-off-by: Joao Martins Signed-off-by: David Woodhouse Signed-off-by: David Woodhouse --- arch/x86/include/asm/kvm_host.h | 13 +++++++ arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/trace.h | 33 +++++++++++++++++ arch/x86/kvm/x86.c | 12 +++++++ arch/x86/kvm/xen.c | 79 +++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/xen.h | 10 ++++++ include/uapi/linux/kvm.h | 17 ++++++++- 7 files changed, 164 insertions(+), 2 deletions(-) create mode 100644 arch/x86/kvm/xen.c create mode 100644 arch/x86/kvm/xen.h diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 9417febf8490..0f469ce439c0 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -79,6 +79,7 @@ #define KVM_REQ_HV_STIMER KVM_ARCH_REQ(22) #define KVM_REQ_LOAD_EOI_EXITMAP KVM_ARCH_REQ(23) #define KVM_REQ_GET_VMCS12_PAGES KVM_ARCH_REQ(24) +#define KVM_REQ_XEN_EXIT KVM_ARCH_REQ(25) #define CR0_RESERVED_BITS \ (~(unsigned long)(X86_CR0_PE | X86_CR0_MP | X86_CR0_EM | X86_CR0_TS \ @@ -533,6 +534,11 @@ struct kvm_vcpu_hv { cpumask_t tlb_flush; }; +/* Xen per vcpu emulation context */ +struct kvm_vcpu_xen { + struct kvm_xen_exit exit; +}; + struct kvm_vcpu_arch { /* * rip and regs accesses must go through @@ -720,6 +726,7 @@ struct kvm_vcpu_arch { unsigned long singlestep_rip; struct kvm_vcpu_hv hyperv; + struct kvm_vcpu_xen xen; cpumask_var_t wbinvd_dirty_mask; @@ -833,6 +840,11 @@ struct kvm_hv { atomic_t num_mismatched_vp_indexes; }; +/* Xen emulation context */ +struct kvm_xen { + u64 xen_hypercall; +}; + enum kvm_irqchip_mode { KVM_IRQCHIP_NONE, KVM_IRQCHIP_KERNEL, /* created with KVM_CREATE_IRQCHIP */ @@ -899,6 +911,7 @@ struct kvm_arch { struct hlist_head mask_notifier_list; struct kvm_hv hyperv; + struct kvm_xen xen; #ifdef CONFIG_KVM_MMU_AUDIT int audit_point; diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile index 31ecf7a76d5a..2b46c93c9380 100644 --- a/arch/x86/kvm/Makefile +++ b/arch/x86/kvm/Makefile @@ -10,7 +10,7 @@ kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o kvm-y += x86.o mmu.o emulate.o i8259.o irq.o lapic.o \ i8254.o ioapic.o irq_comm.o cpuid.o pmu.o mtrr.o \ - hyperv.o page_track.o debugfs.o + hyperv.o xen.o page_track.o debugfs.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o vmx/evmcs.o vmx/nested.o kvm-amd-y += svm.o pmu_amd.o diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h index 6432d08c7de7..4fe9fd86292f 100644 --- a/arch/x86/kvm/trace.h +++ b/arch/x86/kvm/trace.h @@ -91,6 +91,39 @@ TRACE_EVENT(kvm_hv_hypercall, ); /* + * Tracepoint for Xen hypercall. + */ +TRACE_EVENT(kvm_xen_hypercall, + TP_PROTO(unsigned long nr, unsigned long a0, unsigned long a1, + unsigned long a2, unsigned long a3, unsigned long a4), + TP_ARGS(nr, a0, a1, a2, a3, a4), + + TP_STRUCT__entry( + __field(unsigned long, nr) + __field(unsigned long, a0) + __field(unsigned long, a1) + __field(unsigned long, a2) + __field(unsigned long, a3) + __field(unsigned long, a4) + ), + + TP_fast_assign( + __entry->nr = nr; + __entry->a0 = a0; + __entry->a1 = a1; + __entry->a2 = a2; + __entry->a3 = a3; + __entry->a4 = a4; + ), + + TP_printk("nr 0x%lx a0 0x%lx a1 0x%lx a2 0x%lx a3 0x%lx a4 0x%lx", + __entry->nr, __entry->a0, __entry->a1, __entry->a2, + __entry->a3, __entry->a4) +); + + + +/* * Tracepoint for PIO. */ diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 47360a4b0d42..be8def385e3f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -29,6 +29,7 @@ #include "cpuid.h" #include "pmu.h" #include "hyperv.h" +#include "xen.h" #include #include @@ -2338,6 +2339,8 @@ static int xen_hvm_config(struct kvm_vcpu *vcpu, u64 data) } if (kvm_vcpu_write_guest(vcpu, page_addr, page, PAGE_SIZE)) goto out_free; + + kvm_xen_hypercall_set(kvm); r = 0; out_free: kfree(page); @@ -7076,6 +7079,9 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) if (kvm_hv_hypercall_enabled(vcpu->kvm)) return kvm_hv_hypercall(vcpu); + if (kvm_xen_hypercall_enabled(vcpu->kvm)) + return kvm_xen_hypercall(vcpu); + nr = kvm_register_read(vcpu, VCPU_REGS_RAX); a0 = kvm_register_read(vcpu, VCPU_REGS_RBX); a1 = kvm_register_read(vcpu, VCPU_REGS_RCX); @@ -7736,6 +7742,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) r = 0; goto out; } + if (kvm_check_request(KVM_REQ_XEN_EXIT, vcpu)) { + vcpu->run->exit_reason = KVM_EXIT_XEN; + vcpu->run->xen = vcpu->arch.xen.exit; + r = 0; + goto out; + } /* * KVM_REQ_HV_STIMER has to be processed after diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c new file mode 100644 index 000000000000..76f0e4b812d2 --- /dev/null +++ b/arch/x86/kvm/xen.c @@ -0,0 +1,79 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2019 Oracle and/or its affiliates. All rights reserved. + * + * KVM Xen emulation + */ + +#include "x86.h" +#include "xen.h" + +#include + +#include + +#include "trace.h" + +bool kvm_xen_hypercall_enabled(struct kvm *kvm) +{ + return READ_ONCE(kvm->arch.xen.xen_hypercall); +} + +bool kvm_xen_hypercall_set(struct kvm *kvm) +{ + return WRITE_ONCE(kvm->arch.xen.xen_hypercall, 1); +} + +static void kvm_xen_hypercall_set_result(struct kvm_vcpu *vcpu, u64 result) +{ + kvm_register_write(vcpu, VCPU_REGS_RAX, result); +} + +static int kvm_xen_hypercall_complete_userspace(struct kvm_vcpu *vcpu) +{ + struct kvm_run *run = vcpu->run; + + kvm_xen_hypercall_set_result(vcpu, run->xen.u.hcall.result); + return kvm_skip_emulated_instruction(vcpu); +} + +int kvm_xen_hypercall(struct kvm_vcpu *vcpu) +{ + bool longmode; + u64 input, params[5]; + + input = (u64)kvm_register_read(vcpu, VCPU_REGS_RAX); + + longmode = is_64_bit_mode(vcpu); + if (!longmode) { + params[0] = (u64)kvm_register_read(vcpu, VCPU_REGS_RBX); + params[1] = (u64)kvm_register_read(vcpu, VCPU_REGS_RCX); + params[2] = (u64)kvm_register_read(vcpu, VCPU_REGS_RDX); + params[3] = (u64)kvm_register_read(vcpu, VCPU_REGS_RSI); + params[4] = (u64)kvm_register_read(vcpu, VCPU_REGS_RDI); + } +#ifdef CONFIG_X86_64 + else { + params[0] = (u64)kvm_register_read(vcpu, VCPU_REGS_RDI); + params[1] = (u64)kvm_register_read(vcpu, VCPU_REGS_RSI); + params[2] = (u64)kvm_register_read(vcpu, VCPU_REGS_RDX); + params[3] = (u64)kvm_register_read(vcpu, VCPU_REGS_R10); + params[4] = (u64)kvm_register_read(vcpu, VCPU_REGS_R8); + } +#endif + trace_kvm_xen_hypercall(input, params[0], params[1], params[2], + params[3], params[4]); + + vcpu->run->exit_reason = KVM_EXIT_XEN; + vcpu->run->xen.type = KVM_EXIT_XEN_HCALL; + vcpu->run->xen.u.hcall.input = input; + vcpu->run->xen.u.hcall.params[0] = params[0]; + vcpu->run->xen.u.hcall.params[1] = params[1]; + vcpu->run->xen.u.hcall.params[2] = params[2]; + vcpu->run->xen.u.hcall.params[3] = params[3]; + vcpu->run->xen.u.hcall.params[4] = params[4]; + vcpu->arch.complete_userspace_io = + kvm_xen_hypercall_complete_userspace; + + return 0; +} diff --git a/arch/x86/kvm/xen.h b/arch/x86/kvm/xen.h new file mode 100644 index 000000000000..a2ae079c3ef3 --- /dev/null +++ b/arch/x86/kvm/xen.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2019 Oracle and/or its affiliates. All rights reserved. */ +#ifndef __ARCH_X86_KVM_XEN_H__ +#define __ARCH_X86_KVM_XEN_H__ + +bool kvm_xen_hypercall_enabled(struct kvm *kvm); +bool kvm_xen_hypercall_set(struct kvm *kvm); +int kvm_xen_hypercall(struct kvm_vcpu *vcpu); + +#endif diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 6d4ea4b6c922..d07520c216a1 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -204,6 +204,18 @@ struct kvm_hyperv_exit { } u; }; +struct kvm_xen_exit { +#define KVM_EXIT_XEN_HCALL 1 + __u32 type; + union { + struct { + __u64 input; + __u64 result; + __u64 params[5]; + } hcall; + } u; +}; + #define KVM_S390_GET_SKEYS_NONE 1 #define KVM_S390_SKEYS_MAX 1048576 @@ -235,6 +247,7 @@ struct kvm_hyperv_exit { #define KVM_EXIT_S390_STSI 25 #define KVM_EXIT_IOAPIC_EOI 26 #define KVM_EXIT_HYPERV 27 +#define KVM_EXIT_XEN 28 /* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */ @@ -392,8 +405,10 @@ struct kvm_run { } eoi; /* KVM_EXIT_HYPERV */ struct kvm_hyperv_exit hyperv; + /* KVM_EXIT_XEN */ + struct kvm_xen_exit xen; /* Fix the size of the union. */ - char padding[256]; + char padding[196]; }; /* 2048 is the size of the char array used to bound/pad the size From patchwork Wed Feb 20 20:15:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822705 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7CC5E1399 for ; Wed, 20 Feb 2019 20:22:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 691262F360 for ; Wed, 20 Feb 2019 20:22:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5BF922F40B; Wed, 20 Feb 2019 20:22:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C8E032F251 for ; Wed, 20 Feb 2019 20:22:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726876AbfBTURX (ORCPT ); Wed, 20 Feb 2019 15:17:23 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:51706 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725798AbfBTURW (ORCPT ); Wed, 20 Feb 2019 15:17:22 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8jqE081895; Wed, 20 Feb 2019 20:17:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=eN8fngoVKN0x/zeZw68QC5D2VC0C4AAVlV6ydxu23m8=; b=OQ58Zx6cJjilb2OJveOO0gSbbGMIhsRp06cWHLJMujdfmQPw0Il6ZXE1dZjoW08/UH4z npH0YlJUy9hLPAmJWjcNpCnst3s9Hs2oqcJ4my5yqS4b5qhImLQmMMf2IE/EfRWhhxIp FrttRAkiUYTIw/ns/dV1mmeEwSN0Wvc8Ucw0VNueVlHojHK/WielVfLJEFS8k8TwD0dV QkBpSy223mhOk8kL6m435lMvCeSV2SqmwO+1pzvbnmfROaVh2S9Clj5AEL5i7wVFLdSp 8zfcmL+JFn5IJ38WJ93feTENjYs+Ohjo3Mbczb6P1eiGxmxZBPbsUjHTjVDnyw5rGymN tA== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2qp9xu3xe7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:09 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKH35K020565 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:03 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKH3vo018072; Wed, 20 Feb 2019 20:17:03 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:02 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 03/39] KVM: x86/xen: register shared_info page Date: Wed, 20 Feb 2019 20:15:33 +0000 Message-Id: <20190220201609.28290-4-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We add a new ioctl, XEN_HVM_SHARED_INFO, to allow hypervisor to know where the guest's shared info page is. Signed-off-by: Joao Martins --- arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/x86.c | 21 +++++++++++++++ arch/x86/kvm/xen.c | 60 +++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/xen.h | 4 +++ include/uapi/linux/kvm.h | 15 +++++++++++ 5 files changed, 103 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 0f469ce439c0..befc0e37f162 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -843,6 +843,9 @@ struct kvm_hv { /* Xen emulation context */ struct kvm_xen { u64 xen_hypercall; + + gfn_t shinfo_addr; + struct shared_info *shinfo; }; enum kvm_irqchip_mode { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index be8def385e3f..1eda96304180 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4793,6 +4793,26 @@ long kvm_arch_vm_ioctl(struct file *filp, r = 0; break; } + case KVM_XEN_HVM_GET_ATTR: { + struct kvm_xen_hvm_attr xha; + + r = -EFAULT; + if (copy_from_user(&xha, argp, sizeof(xha))) + goto out; + r = kvm_xen_hvm_get_attr(kvm, &xha); + if (copy_to_user(argp, &xha, sizeof(xha))) + goto out; + break; + } + case KVM_XEN_HVM_SET_ATTR: { + struct kvm_xen_hvm_attr xha; + + r = -EFAULT; + if (copy_from_user(&xha, argp, sizeof(xha))) + goto out; + r = kvm_xen_hvm_set_attr(kvm, &xha); + break; + } case KVM_SET_CLOCK: { struct kvm_clock_data user_ns; u64 now_ns; @@ -9279,6 +9299,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm) kvm_mmu_uninit_vm(kvm); kvm_page_track_cleanup(kvm); kvm_hv_destroy_vm(kvm); + kvm_xen_destroy_vm(kvm); } void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free, diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 76f0e4b812d2..4df223bd3cd7 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -11,9 +11,61 @@ #include #include +#include #include "trace.h" +static int kvm_xen_shared_info_init(struct kvm *kvm, gfn_t gfn) +{ + struct shared_info *shared_info; + struct page *page; + + page = gfn_to_page(kvm, gfn); + if (is_error_page(page)) + return -EINVAL; + + kvm->arch.xen.shinfo_addr = gfn; + + shared_info = page_to_virt(page); + memset(shared_info, 0, sizeof(struct shared_info)); + kvm->arch.xen.shinfo = shared_info; + return 0; +} + +int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) +{ + int r = -ENOENT; + + switch (data->type) { + case KVM_XEN_ATTR_TYPE_SHARED_INFO: { + gfn_t gfn = data->u.shared_info.gfn; + + r = kvm_xen_shared_info_init(kvm, gfn); + break; + } + default: + break; + } + + return r; +} + +int kvm_xen_hvm_get_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) +{ + int r = -ENOENT; + + switch (data->type) { + case KVM_XEN_ATTR_TYPE_SHARED_INFO: { + data->u.shared_info.gfn = kvm->arch.xen.shinfo_addr; + break; + } + default: + break; + } + + return r; +} + bool kvm_xen_hypercall_enabled(struct kvm *kvm) { return READ_ONCE(kvm->arch.xen.xen_hypercall); @@ -77,3 +129,11 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) return 0; } + +void kvm_xen_destroy_vm(struct kvm *kvm) +{ + struct kvm_xen *xen = &kvm->arch.xen; + + if (xen->shinfo) + put_page(virt_to_page(xen->shinfo)); +} diff --git a/arch/x86/kvm/xen.h b/arch/x86/kvm/xen.h index a2ae079c3ef3..bb38edf383fe 100644 --- a/arch/x86/kvm/xen.h +++ b/arch/x86/kvm/xen.h @@ -3,8 +3,12 @@ #ifndef __ARCH_X86_KVM_XEN_H__ #define __ARCH_X86_KVM_XEN_H__ +int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data); +int kvm_xen_hvm_get_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data); bool kvm_xen_hypercall_enabled(struct kvm *kvm); bool kvm_xen_hypercall_set(struct kvm *kvm); int kvm_xen_hypercall(struct kvm_vcpu *vcpu); +void kvm_xen_destroy_vm(struct kvm *kvm); + #endif diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index d07520c216a1..de2168d235af 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1455,6 +1455,21 @@ struct kvm_enc_region { /* Available with KVM_CAP_HYPERV_CPUID */ #define KVM_GET_SUPPORTED_HV_CPUID _IOWR(KVMIO, 0xc1, struct kvm_cpuid2) +#define KVM_XEN_HVM_GET_ATTR _IOWR(KVMIO, 0xc2, struct kvm_xen_hvm_attr) +#define KVM_XEN_HVM_SET_ATTR _IOW(KVMIO, 0xc3, struct kvm_xen_hvm_attr) + +struct kvm_xen_hvm_attr { + __u16 type; + + union { + struct { + __u64 gfn; + } shared_info; + } u; +}; + +#define KVM_XEN_ATTR_TYPE_SHARED_INFO 0x0 + /* Secure Encrypted Virtualization command */ enum sev_cmd_id { /* Guest initialization commands */ From patchwork Wed Feb 20 20:15:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822701 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5F2AF922 for ; Wed, 20 Feb 2019 20:21:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4D46B2F28E for ; Wed, 20 Feb 2019 20:21:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 41D682F2B7; Wed, 20 Feb 2019 20:21:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BCE292F28E for ; Wed, 20 Feb 2019 20:21:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727096AbfBTUR1 (ORCPT ); Wed, 20 Feb 2019 15:17:27 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:51792 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727053AbfBTUR0 (ORCPT ); Wed, 20 Feb 2019 15:17:26 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8UCE081678; Wed, 20 Feb 2019 20:17:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=al+fO0YM+/IdPORa/NVtvyagkKve8vydINcMaFyRhuU=; b=ZK0gbkf7s34driGkAUZ6IKo9bt2YJud67HsrMiWjq72VI6Hou+zfqXodVdylQU07SOJp +vStemVsDTYrOEiTLa34xH2jtwDl0qh0Ax/0Xt7Qjd3yO2y5csDizXq7Ct1n7Gu5s1X6 PhIq2+3s+B/cb2DmDBkh6hO1EahfgK0rBCmiOvk5J0ur26JKSY9lMCBcDOCnwQahrKBW cSyDVLxziDoo5P5txKHXdGFc3mprc1ZheFrmtDGThHkKpIUJXGXjjXu0RA3+H4aQeHfF ozzyfNex0mtxsl9EvZygl3LM/PIcf7mYl7Ld05fze8n3qYzeHasO2w/mWiNNExZGDjPb Rg== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2qp9xu3xeh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:13 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKH7nP020701 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:07 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKH6U7018096; Wed, 20 Feb 2019 20:17:06 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:05 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 04/39] KVM: x86/xen: setup pvclock updates Date: Wed, 20 Feb 2019 20:15:34 +0000 Message-Id: <20190220201609.28290-5-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This means when we set shared_info page GPA, and request a master clock update. This will trigger all vcpus to update their respective shared pvclock data with guests. We follow a similar approach as Hyper-V and KVM and adjust it accordingly. Note however that Xen differs a little on how pvclock pages are set up. Specifically KVM assumes 4K page alignment and pvclock data starts in the beginning of the page. Whereas Xen you can place that information anywhere in the page. Signed-off-by: Joao Martins --- arch/x86/kvm/x86.c | 2 ++ arch/x86/kvm/xen.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/xen.h | 1 + 3 files changed, 50 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 1eda96304180..6eb2afaa2af2 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2211,6 +2211,8 @@ static int kvm_guest_time_update(struct kvm_vcpu *v) if (vcpu->pv_time_enabled) kvm_setup_pvclock_page(v); + if (ka->xen.shinfo) + kvm_xen_setup_pvclock_page(v); if (v == kvm_get_vcpu(v->kvm, 0)) kvm_hv_setup_tsc_page(v->kvm, &vcpu->hv_clock); return 0; diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 4df223bd3cd7..b4bd1949656e 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -29,9 +29,56 @@ static int kvm_xen_shared_info_init(struct kvm *kvm, gfn_t gfn) shared_info = page_to_virt(page); memset(shared_info, 0, sizeof(struct shared_info)); kvm->arch.xen.shinfo = shared_info; + + kvm_make_all_cpus_request(kvm, KVM_REQ_MASTERCLOCK_UPDATE); return 0; } +void kvm_xen_setup_pvclock_page(struct kvm_vcpu *v) +{ + struct kvm_vcpu_arch *vcpu = &v->arch; + struct pvclock_vcpu_time_info *guest_hv_clock; + unsigned int offset; + + if (v->vcpu_id >= MAX_VIRT_CPUS) + return; + + offset = offsetof(struct vcpu_info, time); + offset += offsetof(struct shared_info, vcpu_info); + offset += v->vcpu_id * sizeof(struct vcpu_info); + + guest_hv_clock = (struct pvclock_vcpu_time_info *) + (((void *)v->kvm->arch.xen.shinfo) + offset); + + BUILD_BUG_ON(offsetof(struct pvclock_vcpu_time_info, version) != 0); + + if (guest_hv_clock->version & 1) + ++guest_hv_clock->version; /* first time write, random junk */ + + vcpu->hv_clock.version = guest_hv_clock->version + 1; + guest_hv_clock->version = vcpu->hv_clock.version; + + smp_wmb(); + + /* retain PVCLOCK_GUEST_STOPPED if set in guest copy */ + vcpu->hv_clock.flags |= (guest_hv_clock->flags & PVCLOCK_GUEST_STOPPED); + + if (vcpu->pvclock_set_guest_stopped_request) { + vcpu->hv_clock.flags |= PVCLOCK_GUEST_STOPPED; + vcpu->pvclock_set_guest_stopped_request = false; + } + + trace_kvm_pvclock_update(v->vcpu_id, &vcpu->hv_clock); + + *guest_hv_clock = vcpu->hv_clock; + + smp_wmb(); + + vcpu->hv_clock.version++; + + guest_hv_clock->version = vcpu->hv_clock.version; +} + int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) { int r = -ENOENT; diff --git a/arch/x86/kvm/xen.h b/arch/x86/kvm/xen.h index bb38edf383fe..827c9390da34 100644 --- a/arch/x86/kvm/xen.h +++ b/arch/x86/kvm/xen.h @@ -3,6 +3,7 @@ #ifndef __ARCH_X86_KVM_XEN_H__ #define __ARCH_X86_KVM_XEN_H__ +void kvm_xen_setup_pvclock_page(struct kvm_vcpu *vcpu); int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data); int kvm_xen_hvm_get_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data); bool kvm_xen_hypercall_enabled(struct kvm *kvm); From patchwork Wed Feb 20 20:15:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822703 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 86DC41399 for ; Wed, 20 Feb 2019 20:21:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 736DF2F0E8 for ; Wed, 20 Feb 2019 20:21:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 715D92F360; Wed, 20 Feb 2019 20:21:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17F2B2F2B7 for ; Wed, 20 Feb 2019 20:21:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726953AbfBTURY (ORCPT ); Wed, 20 Feb 2019 15:17:24 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:34888 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726602AbfBTURY (ORCPT ); Wed, 20 Feb 2019 15:17:24 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8ceC087610; Wed, 20 Feb 2019 20:17:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=V5q13HBn+4ZOF9QMLFs45AmHfiywEMNQk6g5OyP6QrY=; b=hVyhkVLIZA9ED2oyH5lx+R4dLdbhfDwfHVu1ctmirGljINrZQNNVP7wNkNcUdlJeyeLq Gvoc9LO4n6uwFbk/lYzh378RuYSwXNuevLEo4DLS2tFtoZRiSXlD5FWiDNAMFE9eIU/J Eg0xNPdxjLfFOItq7Rb9wC/avlJNb9JpMisxH0QavqWdYJgLZCv5fLcA3hBzGjDAcem0 s/9QL0AhzZyUYOJ0IPBg9BU4o7OASLwFf/C2orP3eRurk2If/b86DvE9yFt7VelazjHu SZ9Gz77wHeBS7wBEQ/dQuQ0Zw0pyhOWu7DI+55F4h7tLm4cadX/Xzg1aQLP/Ndlq2m11 Hg== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2qpb5rktna-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:10 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHAt1025009 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:10 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKH9OM011752; Wed, 20 Feb 2019 20:17:09 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:08 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 05/39] KVM: x86/xen: update wallclock region Date: Wed, 20 Feb 2019 20:15:35 +0000 Message-Id: <20190220201609.28290-6-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=748 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Wallclock on Xen is written in the shared_info page. To that purpose, export kvm_write_wall_clock() and pass on the GPA of its location to populate the shared_info wall clock data. Signed-off-by: Joao Martins --- arch/x86/kvm/x86.c | 2 +- arch/x86/kvm/x86.h | 1 + arch/x86/kvm/xen.c | 3 +++ 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 6eb2afaa2af2..31a102b22042 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1418,7 +1418,7 @@ void kvm_set_pending_timer(struct kvm_vcpu *vcpu) kvm_make_request(KVM_REQ_PENDING_TIMER, vcpu); } -static void kvm_write_wall_clock(struct kvm *kvm, gpa_t wall_clock) +void kvm_write_wall_clock(struct kvm *kvm, gpa_t wall_clock) { int version; int r; diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 28406aa1136d..b5f2e66a4c81 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -256,6 +256,7 @@ static inline bool kvm_check_has_quirk(struct kvm *kvm, u64 quirk) } void kvm_set_pending_timer(struct kvm_vcpu *vcpu); +void kvm_write_wall_clock(struct kvm *kvm, gpa_t wall_clock); int kvm_inject_realmode_interrupt(struct kvm_vcpu *vcpu, int irq, int inc_eip); void kvm_write_tsc(struct kvm_vcpu *vcpu, struct msr_data *msr); diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index b4bd1949656e..879bcfdd7b1d 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -19,6 +19,7 @@ static int kvm_xen_shared_info_init(struct kvm *kvm, gfn_t gfn) { struct shared_info *shared_info; struct page *page; + gpa_t gpa = gfn_to_gpa(gfn); page = gfn_to_page(kvm, gfn); if (is_error_page(page)) @@ -30,6 +31,8 @@ static int kvm_xen_shared_info_init(struct kvm *kvm, gfn_t gfn) memset(shared_info, 0, sizeof(struct shared_info)); kvm->arch.xen.shinfo = shared_info; + kvm_write_wall_clock(kvm, gpa + offsetof(struct shared_info, wc)); + kvm_make_all_cpus_request(kvm, KVM_REQ_MASTERCLOCK_UPDATE); return 0; } From patchwork Wed Feb 20 20:15:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822699 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 471751399 for ; Wed, 20 Feb 2019 20:21:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C7322F28B for ; Wed, 20 Feb 2019 20:21:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 20FBD2F198; Wed, 20 Feb 2019 20:21:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7BEF92F198 for ; Wed, 20 Feb 2019 20:21:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727157AbfBTURa (ORCPT ); Wed, 20 Feb 2019 15:17:30 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:51870 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727139AbfBTUR3 (ORCPT ); Wed, 20 Feb 2019 15:17:29 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK9MH4082471; Wed, 20 Feb 2019 20:17:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=AsFZ89mGwV06Z97LV9y48tpzgtmNIVqvwCB8HHS2rLA=; b=R0Unt9bzHsc1ntgg+HuP/PIoGD0PQFUpjULiLUxRuhsqJeGyE3IakUBfvJ5/xkMTY9sB 27ySCxjRfILQiMOVs6nYet6igQkiF5zCi0DSR2uGE6M39nQLhCHAxCdWuHiN7xV19Ev3 476au636pSBip1Rwcbj4TbX7s2ULKgXNO19XXJChmZYwAoubvyflXE+Oj3IFbtxQdBkJ rPSDYcMoV77FrICZSpQI/O/eYm4+4qA2qsL510A42j0t3b7xBeMinXouh0AIYAkRpIBx Ga/nb96J/mYDMxJzg2XduubDoKeYuw6XPDZ8AQFmN5gue1Zni78zqtkQz6tFnKABBabp 5A== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2qp9xu3xeq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:15 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHDdm021057 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:14 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKHDXH015354; Wed, 20 Feb 2019 20:17:13 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:12 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 06/39] KVM: x86/xen: register vcpu info Date: Wed, 20 Feb 2019 20:15:36 +0000 Message-Id: <20190220201609.28290-7-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The vcpu info supersedes the per vcpu area of the shared info page and the guest vcpus will use this instead. Signed-off-by: Joao Martins Signed-off-by: Ankur Arora --- arch/x86/include/asm/kvm_host.h | 2 + arch/x86/kvm/x86.c | 1 + arch/x86/kvm/xen.c | 93 ++++++++++++++++++++++++++++++++++++++--- arch/x86/kvm/xen.h | 14 +++++++ include/uapi/linux/kvm.h | 5 +++ 5 files changed, 110 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index befc0e37f162..96f65ba4b3c0 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -537,6 +537,8 @@ struct kvm_vcpu_hv { /* Xen per vcpu emulation context */ struct kvm_vcpu_xen { struct kvm_xen_exit exit; + gpa_t vcpu_info_addr; + struct vcpu_info *vcpu_info; }; struct kvm_vcpu_arch { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 31a102b22042..3ce97860e6ee 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9124,6 +9124,7 @@ void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) int idx; kvm_hv_vcpu_uninit(vcpu); + kvm_xen_vcpu_uninit(vcpu); kvm_pmu_destroy(vcpu); kfree(vcpu->arch.mce_banks); kvm_free_lapic(vcpu); diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 879bcfdd7b1d..36d6dd0ea4b8 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -15,6 +15,33 @@ #include "trace.h" +static void set_vcpu_attr(struct kvm_vcpu *v, u16 type, gpa_t gpa, void *addr) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(v); + + switch (type) { + case KVM_XEN_ATTR_TYPE_VCPU_INFO: + vcpu_xen->vcpu_info_addr = gpa; + vcpu_xen->vcpu_info = addr; + kvm_xen_setup_pvclock_page(v); + break; + default: + break; + } +} + +static gpa_t get_vcpu_attr(struct kvm_vcpu *v, u16 type) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(v); + + switch (type) { + case KVM_XEN_ATTR_TYPE_VCPU_INFO: + return vcpu_xen->vcpu_info_addr; + default: + return 0; + } +} + static int kvm_xen_shared_info_init(struct kvm *kvm, gfn_t gfn) { struct shared_info *shared_info; @@ -37,26 +64,44 @@ static int kvm_xen_shared_info_init(struct kvm *kvm, gfn_t gfn) return 0; } +static void *xen_vcpu_info(struct kvm_vcpu *v) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(v); + struct kvm_xen *kvm = &v->kvm->arch.xen; + unsigned int offset = 0; + void *hva = NULL; + + if (vcpu_xen->vcpu_info_addr) + return vcpu_xen->vcpu_info; + + if (kvm->shinfo_addr && v->vcpu_id < MAX_VIRT_CPUS) { + hva = kvm->shinfo; + offset += offsetof(struct shared_info, vcpu_info); + offset += v->vcpu_id * sizeof(struct vcpu_info); + } + + return hva + offset; +} + void kvm_xen_setup_pvclock_page(struct kvm_vcpu *v) { struct kvm_vcpu_arch *vcpu = &v->arch; struct pvclock_vcpu_time_info *guest_hv_clock; + void *hva = xen_vcpu_info(v); unsigned int offset; - if (v->vcpu_id >= MAX_VIRT_CPUS) + if (!hva) return; offset = offsetof(struct vcpu_info, time); - offset += offsetof(struct shared_info, vcpu_info); - offset += v->vcpu_id * sizeof(struct vcpu_info); guest_hv_clock = (struct pvclock_vcpu_time_info *) - (((void *)v->kvm->arch.xen.shinfo) + offset); + (hva + offset); BUILD_BUG_ON(offsetof(struct pvclock_vcpu_time_info, version) != 0); if (guest_hv_clock->version & 1) - ++guest_hv_clock->version; /* first time write, random junk */ + ++guest_hv_clock->version; vcpu->hv_clock.version = guest_hv_clock->version + 1; guest_hv_clock->version = vcpu->hv_clock.version; @@ -93,6 +138,25 @@ int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) r = kvm_xen_shared_info_init(kvm, gfn); break; } + case KVM_XEN_ATTR_TYPE_VCPU_INFO: { + gpa_t gpa = data->u.vcpu_attr.gpa; + struct kvm_vcpu *v; + struct page *page; + void *addr; + + v = kvm_get_vcpu(kvm, data->u.vcpu_attr.vcpu); + if (!v) + return -EINVAL; + + page = gfn_to_page(v->kvm, gpa_to_gfn(gpa)); + if (is_error_page(page)) + return -EFAULT; + + addr = page_to_virt(page) + offset_in_page(gpa); + set_vcpu_attr(v, data->type, gpa, addr); + r = 0; + break; + } default: break; } @@ -109,6 +173,17 @@ int kvm_xen_hvm_get_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) data->u.shared_info.gfn = kvm->arch.xen.shinfo_addr; break; } + case KVM_XEN_ATTR_TYPE_VCPU_INFO: { + struct kvm_vcpu *v; + + v = kvm_get_vcpu(kvm, data->u.vcpu_attr.vcpu); + if (!v) + return -EINVAL; + + data->u.vcpu_attr.gpa = get_vcpu_attr(v, data->type); + r = 0; + break; + } default: break; } @@ -180,6 +255,14 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) return 0; } +void kvm_xen_vcpu_uninit(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(vcpu); + + if (vcpu_xen->vcpu_info) + put_page(virt_to_page(vcpu_xen->vcpu_info)); +} + void kvm_xen_destroy_vm(struct kvm *kvm) { struct kvm_xen *xen = &kvm->arch.xen; diff --git a/arch/x86/kvm/xen.h b/arch/x86/kvm/xen.h index 827c9390da34..10ebd0b7a25e 100644 --- a/arch/x86/kvm/xen.h +++ b/arch/x86/kvm/xen.h @@ -3,6 +3,19 @@ #ifndef __ARCH_X86_KVM_XEN_H__ #define __ARCH_X86_KVM_XEN_H__ +static inline struct kvm_vcpu_xen *vcpu_to_xen_vcpu(struct kvm_vcpu *vcpu) +{ + return &vcpu->arch.xen; +} + +static inline struct kvm_vcpu *xen_vcpu_to_vcpu(struct kvm_vcpu_xen *xen_vcpu) +{ + struct kvm_vcpu_arch *arch; + + arch = container_of(xen_vcpu, struct kvm_vcpu_arch, xen); + return container_of(arch, struct kvm_vcpu, arch); +} + void kvm_xen_setup_pvclock_page(struct kvm_vcpu *vcpu); int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data); int kvm_xen_hvm_get_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data); @@ -11,5 +24,6 @@ bool kvm_xen_hypercall_set(struct kvm *kvm); int kvm_xen_hypercall(struct kvm_vcpu *vcpu); void kvm_xen_destroy_vm(struct kvm *kvm); +void kvm_xen_vcpu_uninit(struct kvm_vcpu *vcpu); #endif diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index de2168d235af..782f497a0fdd 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1465,10 +1465,15 @@ struct kvm_xen_hvm_attr { struct { __u64 gfn; } shared_info; + struct { + __u32 vcpu; + __u64 gpa; + } vcpu_attr; } u; }; #define KVM_XEN_ATTR_TYPE_SHARED_INFO 0x0 +#define KVM_XEN_ATTR_TYPE_VCPU_INFO 0x1 /* Secure Encrypted Virtualization command */ enum sev_cmd_id { From patchwork Wed Feb 20 20:15:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822695 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CBC7F922 for ; Wed, 20 Feb 2019 20:21:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B74FE2F05B for ; Wed, 20 Feb 2019 20:21:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AB48E2F251; Wed, 20 Feb 2019 20:21:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 201E02F212 for ; Wed, 20 Feb 2019 20:21:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727401AbfBTUVb (ORCPT ); Wed, 20 Feb 2019 15:21:31 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:60974 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727192AbfBTURb (ORCPT ); Wed, 20 Feb 2019 15:17:31 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK95Wd088088; Wed, 20 Feb 2019 20:17:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=l7fb/dXFfaQBZgqf9ui2KpVrBMWCvGVoCEVnuwIloQc=; b=ZqLyANRgYj0arQDEs4AfnZcFy6M1QcRyNS3X58r54VNyflHV/wcHNPQu4a14VRpw0vei i2d8cWodYI+6PsHzc2m4HzXOGVk3Z50+9ihnLhFinYntlMAAWBPVbHo5hfWx+QAH4DCo ea9cNRb9Jc6zoPCpGEB4bPn03s+dTc7qHS0OARPvTS75LEjOsFbuDzRA1U5J2bjHDFRv 0VI411AbSZx9ddrU3frJHkgL/QdeHZJys+LJQaCN3RtVfw9Hp8H96/HxSJ4PKjDsqfNU wRsPPh/VFhub7g960D+ouSOn1JMpOdTRUS8K26c0FHEHcdq4XeS3t746U3o94Pa5J4Pg 8g== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp2130.oracle.com with ESMTP id 2qp81ec382-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:17 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHGra015952 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:16 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHGMi009307; Wed, 20 Feb 2019 20:17:16 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:15 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 07/39] KVM: x86/xen: register vcpu time info region Date: Wed, 20 Feb 2019 20:15:37 +0000 Message-Id: <20190220201609.28290-8-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Allow the Xen emulated guest the ability to register secondary vcpu time information. On Xen guests this is used in order to be mapped to userspace and hence allow vdso gettimeofday to work. In doing so, move kvm_xen_set_pvclock_page() logic to kvm_xen_update_vcpu_time() and have the former a top-level function which updates primary vcpu time info (in struct vcpu_info) and secondary one. Signed-off-by: Joao Martins --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/xen.c | 45 ++++++++++++++++++++++++++++++----------- include/uapi/linux/kvm.h | 1 + 3 files changed, 36 insertions(+), 12 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 96f65ba4b3c0..f39d50dd8f40 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -539,6 +539,8 @@ struct kvm_vcpu_xen { struct kvm_xen_exit exit; gpa_t vcpu_info_addr; struct vcpu_info *vcpu_info; + gpa_t pv_time_addr; + struct pvclock_vcpu_time_info *pv_time; }; struct kvm_vcpu_arch { diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 36d6dd0ea4b8..77d1153386bc 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -25,6 +25,11 @@ static void set_vcpu_attr(struct kvm_vcpu *v, u16 type, gpa_t gpa, void *addr) vcpu_xen->vcpu_info = addr; kvm_xen_setup_pvclock_page(v); break; + case KVM_XEN_ATTR_TYPE_VCPU_TIME_INFO: + vcpu_xen->pv_time_addr = gpa; + vcpu_xen->pv_time = addr; + kvm_xen_setup_pvclock_page(v); + break; default: break; } @@ -37,6 +42,8 @@ static gpa_t get_vcpu_attr(struct kvm_vcpu *v, u16 type) switch (type) { case KVM_XEN_ATTR_TYPE_VCPU_INFO: return vcpu_xen->vcpu_info_addr; + case KVM_XEN_ATTR_TYPE_VCPU_TIME_INFO: + return vcpu_xen->pv_time_addr; default: return 0; } @@ -83,20 +90,10 @@ static void *xen_vcpu_info(struct kvm_vcpu *v) return hva + offset; } -void kvm_xen_setup_pvclock_page(struct kvm_vcpu *v) +static void kvm_xen_update_vcpu_time(struct kvm_vcpu *v, + struct pvclock_vcpu_time_info *guest_hv_clock) { struct kvm_vcpu_arch *vcpu = &v->arch; - struct pvclock_vcpu_time_info *guest_hv_clock; - void *hva = xen_vcpu_info(v); - unsigned int offset; - - if (!hva) - return; - - offset = offsetof(struct vcpu_info, time); - - guest_hv_clock = (struct pvclock_vcpu_time_info *) - (hva + offset); BUILD_BUG_ON(offsetof(struct pvclock_vcpu_time_info, version) != 0); @@ -127,6 +124,26 @@ void kvm_xen_setup_pvclock_page(struct kvm_vcpu *v) guest_hv_clock->version = vcpu->hv_clock.version; } +void kvm_xen_setup_pvclock_page(struct kvm_vcpu *v) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(v); + struct pvclock_vcpu_time_info *guest_hv_clock; + void *hva = xen_vcpu_info(v); + unsigned int offset; + + offset = offsetof(struct vcpu_info, time); + guest_hv_clock = (struct pvclock_vcpu_time_info *) (hva + offset); + + if (likely(hva)) + kvm_xen_update_vcpu_time(v, guest_hv_clock); + + /* Update secondary pvclock region if registered */ + if (vcpu_xen->pv_time_addr) { + guest_hv_clock = vcpu_xen->pv_time; + kvm_xen_update_vcpu_time(v, guest_hv_clock); + } +} + int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) { int r = -ENOENT; @@ -138,6 +155,7 @@ int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) r = kvm_xen_shared_info_init(kvm, gfn); break; } + case KVM_XEN_ATTR_TYPE_VCPU_TIME_INFO: case KVM_XEN_ATTR_TYPE_VCPU_INFO: { gpa_t gpa = data->u.vcpu_attr.gpa; struct kvm_vcpu *v; @@ -173,6 +191,7 @@ int kvm_xen_hvm_get_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) data->u.shared_info.gfn = kvm->arch.xen.shinfo_addr; break; } + case KVM_XEN_ATTR_TYPE_VCPU_TIME_INFO: case KVM_XEN_ATTR_TYPE_VCPU_INFO: { struct kvm_vcpu *v; @@ -261,6 +280,8 @@ void kvm_xen_vcpu_uninit(struct kvm_vcpu *vcpu) if (vcpu_xen->vcpu_info) put_page(virt_to_page(vcpu_xen->vcpu_info)); + if (vcpu_xen->pv_time) + put_page(virt_to_page(vcpu_xen->pv_time)); } void kvm_xen_destroy_vm(struct kvm *kvm) diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 782f497a0fdd..8296c3a2434f 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1474,6 +1474,7 @@ struct kvm_xen_hvm_attr { #define KVM_XEN_ATTR_TYPE_SHARED_INFO 0x0 #define KVM_XEN_ATTR_TYPE_VCPU_INFO 0x1 +#define KVM_XEN_ATTR_TYPE_VCPU_TIME_INFO 0x2 /* Secure Encrypted Virtualization command */ enum sev_cmd_id { From patchwork Wed Feb 20 20:15:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822693 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D293A1399 for ; Wed, 20 Feb 2019 20:21:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BFD012F142 for ; Wed, 20 Feb 2019 20:21:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B3EBC2F1F9; Wed, 20 Feb 2019 20:21:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 200DC2F044 for ; Wed, 20 Feb 2019 20:21:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727319AbfBTURg (ORCPT ); Wed, 20 Feb 2019 15:17:36 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:51976 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727263AbfBTURe (ORCPT ); Wed, 20 Feb 2019 15:17:34 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8Xht081717; Wed, 20 Feb 2019 20:17:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=mdHWAv+RFA17VCCI7IU4ksER0LZyfmkASS+wZz/Vd28=; b=IeY3iumGqJMHu1u0Dgpp70e6hjbnzUO87NWuFoujPczT/YPGp7LyH1sZXWLjBuiiHhCJ o1JuyVRPdk0j/c8ez9dL/881mDuZiX+E6PUG3o6zze7s1A/plGo2cIol3U2Cq4bno9sN CjjMylzEoFYgVoQw3ELwpo/BOPvhHZgc4+jUi9YdayvVMm8dCuZljkn4Vicx5AME2hdd QdVafRCPaSauXTYXRq+C88DEjFxi16j156kCgOp5l21uLMWkeg9Vwj3UjzySinRiVVN3 yqdCde4bxHIZ1YvGwNAXgMaeAkmYz46YDv+/efmk9jn08pHIW5LWLT21qDCz/pyCub3w Hw== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2130.oracle.com with ESMTP id 2qp9xu3xf1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:20 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHJr3025389 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:20 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHJPi011792; Wed, 20 Feb 2019 20:17:19 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:19 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 08/39] KVM: x86/xen: register steal clock Date: Wed, 20 Feb 2019 20:15:38 +0000 Message-Id: <20190220201609.28290-9-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Allow emulator to register vcpu runstates which allow Xen guests to use that for steal clock. The 'preempted' state of KVM steal clock equates to 'runnable' state, 'running' has similar meanings for both and 'offline' is used when system admin needs to bring vcpu offline or hotplug. Signed-off-by: Joao Martins --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/x86.c | 10 ++++++++ arch/x86/kvm/xen.c | 51 +++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/xen.h | 2 ++ include/uapi/linux/kvm.h | 1 + 5 files changed, 66 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index f39d50dd8f40..9d388ba0a05c 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -541,6 +541,8 @@ struct kvm_vcpu_xen { struct vcpu_info *vcpu_info; gpa_t pv_time_addr; struct pvclock_vcpu_time_info *pv_time; + gpa_t steal_time_addr; + struct vcpu_runstate_info *steal_time; }; struct kvm_vcpu_arch { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 3ce97860e6ee..888598fdf543 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2389,6 +2389,11 @@ static void kvm_vcpu_flush_tlb(struct kvm_vcpu *vcpu, bool invalidate_gpa) static void record_steal_time(struct kvm_vcpu *vcpu) { + if (vcpu->arch.xen.steal_time_addr) { + kvm_xen_setup_runstate_page(vcpu); + return; + } + if (!(vcpu->arch.st.msr_val & KVM_MSR_ENABLED)) return; @@ -3251,6 +3256,11 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) static void kvm_steal_time_set_preempted(struct kvm_vcpu *vcpu) { + if (vcpu->arch.xen.steal_time_addr) { + kvm_xen_runstate_set_preempted(vcpu); + return; + } + if (!(vcpu->arch.st.msr_val & KVM_MSR_ENABLED)) return; diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 77d1153386bc..4fdc4c71245a 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -9,9 +9,11 @@ #include "xen.h" #include +#include #include #include +#include #include "trace.h" @@ -30,6 +32,11 @@ static void set_vcpu_attr(struct kvm_vcpu *v, u16 type, gpa_t gpa, void *addr) vcpu_xen->pv_time = addr; kvm_xen_setup_pvclock_page(v); break; + case KVM_XEN_ATTR_TYPE_VCPU_RUNSTATE: + vcpu_xen->steal_time_addr = gpa; + vcpu_xen->steal_time = addr; + kvm_xen_setup_runstate_page(v); + break; default: break; } @@ -44,6 +51,8 @@ static gpa_t get_vcpu_attr(struct kvm_vcpu *v, u16 type) return vcpu_xen->vcpu_info_addr; case KVM_XEN_ATTR_TYPE_VCPU_TIME_INFO: return vcpu_xen->pv_time_addr; + case KVM_XEN_ATTR_TYPE_VCPU_RUNSTATE: + return vcpu_xen->steal_time_addr; default: return 0; } @@ -124,6 +133,41 @@ static void kvm_xen_update_vcpu_time(struct kvm_vcpu *v, guest_hv_clock->version = vcpu->hv_clock.version; } +void kvm_xen_runstate_set_preempted(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(vcpu); + int state = RUNSTATE_runnable; + + vcpu->arch.st.steal.preempted = KVM_VCPU_PREEMPTED; + + vcpu_xen->steal_time->state = state; +} + +void kvm_xen_setup_runstate_page(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(vcpu); + struct vcpu_runstate_info runstate; + + runstate = *vcpu_xen->steal_time; + + runstate.state_entry_time += 1; + runstate.state_entry_time |= XEN_RUNSTATE_UPDATE; + vcpu_xen->steal_time->state_entry_time = runstate.state_entry_time; + smp_wmb(); + + vcpu->arch.st.steal.steal += current->sched_info.run_delay - + vcpu->arch.st.last_steal; + vcpu->arch.st.last_steal = current->sched_info.run_delay; + + runstate.state = RUNSTATE_running; + runstate.time[RUNSTATE_runnable] = vcpu->arch.st.steal.steal; + *vcpu_xen->steal_time = runstate; + + runstate.state_entry_time &= ~XEN_RUNSTATE_UPDATE; + vcpu_xen->steal_time->state_entry_time = runstate.state_entry_time; + smp_wmb(); +} + void kvm_xen_setup_pvclock_page(struct kvm_vcpu *v) { struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(v); @@ -155,6 +199,10 @@ int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) r = kvm_xen_shared_info_init(kvm, gfn); break; } + case KVM_XEN_ATTR_TYPE_VCPU_RUNSTATE: + if (unlikely(!sched_info_on())) + return -ENOTSUPP; + /* fallthrough */ case KVM_XEN_ATTR_TYPE_VCPU_TIME_INFO: case KVM_XEN_ATTR_TYPE_VCPU_INFO: { gpa_t gpa = data->u.vcpu_attr.gpa; @@ -191,6 +239,7 @@ int kvm_xen_hvm_get_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) data->u.shared_info.gfn = kvm->arch.xen.shinfo_addr; break; } + case KVM_XEN_ATTR_TYPE_VCPU_RUNSTATE: case KVM_XEN_ATTR_TYPE_VCPU_TIME_INFO: case KVM_XEN_ATTR_TYPE_VCPU_INFO: { struct kvm_vcpu *v; @@ -282,6 +331,8 @@ void kvm_xen_vcpu_uninit(struct kvm_vcpu *vcpu) put_page(virt_to_page(vcpu_xen->vcpu_info)); if (vcpu_xen->pv_time) put_page(virt_to_page(vcpu_xen->pv_time)); + if (vcpu_xen->steal_time) + put_page(virt_to_page(vcpu_xen->steal_time)); } void kvm_xen_destroy_vm(struct kvm *kvm) diff --git a/arch/x86/kvm/xen.h b/arch/x86/kvm/xen.h index 10ebd0b7a25e..2feef68ee80f 100644 --- a/arch/x86/kvm/xen.h +++ b/arch/x86/kvm/xen.h @@ -17,6 +17,8 @@ static inline struct kvm_vcpu *xen_vcpu_to_vcpu(struct kvm_vcpu_xen *xen_vcpu) } void kvm_xen_setup_pvclock_page(struct kvm_vcpu *vcpu); +void kvm_xen_setup_runstate_page(struct kvm_vcpu *vcpu); +void kvm_xen_runstate_set_preempted(struct kvm_vcpu *vcpu); int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data); int kvm_xen_hvm_get_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data); bool kvm_xen_hypercall_enabled(struct kvm *kvm); diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 8296c3a2434f..b91e57d9e6d3 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1475,6 +1475,7 @@ struct kvm_xen_hvm_attr { #define KVM_XEN_ATTR_TYPE_SHARED_INFO 0x0 #define KVM_XEN_ATTR_TYPE_VCPU_INFO 0x1 #define KVM_XEN_ATTR_TYPE_VCPU_TIME_INFO 0x2 +#define KVM_XEN_ATTR_TYPE_VCPU_RUNSTATE 0x3 /* Secure Encrypted Virtualization command */ enum sev_cmd_id { From patchwork Wed Feb 20 20:15:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822691 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 956D01399 for ; Wed, 20 Feb 2019 20:21:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 83B6D2F0E8 for ; Wed, 20 Feb 2019 20:21:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 781612F1F3; Wed, 20 Feb 2019 20:21:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 047392F170 for ; Wed, 20 Feb 2019 20:21:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727389AbfBTURl (ORCPT ); Wed, 20 Feb 2019 15:17:41 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:35116 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727294AbfBTURh (ORCPT ); Wed, 20 Feb 2019 15:17:37 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8X04087279; Wed, 20 Feb 2019 20:17:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=f2LOq6B0XuNZKZBzoY6FcLJqthrRVeHYrz/V57Db200=; b=hphEvXpQvvu/TvmmgwOlbxm3ToiZS1n4SWX6Lch6HFUhfFq0x4eKmRRslA4RuNwUgvrm rfKOhoKAvZMvrBILMVyW8SX0/qxz+1YyhhoDOHZIQSkVS3w+biI6Vha2XjG8jEF2BRV5 4mU32lOg0tL4p7oxDIpAvY1lQxG4Rv7EO6DBPNxmciDu+rVFbTnTtqasey6b4NRxRpc6 3B4gQbUCQ8XXqLqb7ej+f0bYa5q1Bb6LjwyclKW/f5Yp61XIGzkMOuOupaXLQJ3skux8 WP7ZzdK181aD91pwRBD5aKEDG8cmbcVWCF+PeWF+YgvDkRqVaG8m6UF/pBhbKgEX6cM1 5A== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2qpb5rktph-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:23 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHMNa025505 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:22 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHMFB009375; Wed, 20 Feb 2019 20:17:22 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:22 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 09/39] KVM: x86: declare Xen HVM guest capability Date: Wed, 20 Feb 2019 20:15:39 +0000 Message-Id: <20190220201609.28290-10-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=715 adultscore=3 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Also take the chance to document this rather old capability of KVM_CAP_XEN_HVM which only means it supports the Xen hypercall MSR. Signed-off-by: Joao Martins --- Documentation/virtual/kvm/api.txt | 14 ++++++++++++++ arch/x86/kvm/x86.c | 1 + include/uapi/linux/kvm.h | 3 +++ 3 files changed, 18 insertions(+) diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt index 356156f5c52d..c3a1402b76bf 100644 --- a/Documentation/virtual/kvm/api.txt +++ b/Documentation/virtual/kvm/api.txt @@ -5016,6 +5016,7 @@ CPU when the exception is taken. If this virtual SError is taken to EL1 using AArch64, this value will be reported in the ISS field of ESR_ELx. See KVM_CAP_VCPU_EVENTS for more details. + 8.20 KVM_CAP_HYPERV_SEND_IPI Architectures: x86 @@ -5023,3 +5024,16 @@ Architectures: x86 This capability indicates that KVM supports paravirtualized Hyper-V IPI send hypercalls: HvCallSendSyntheticClusterIpi, HvCallSendSyntheticClusterIpiEx. + +8.21 KVM_CAP_XEN_HVM + +Architectures: x86 + +This capability indicates that KVM supports the Xen hypercall page MSR. + +8.22 KVM_CAP_XEN_HVM_GUEST + +Architectures: x86 + +This capability indicates that KVM supports Xen HVM guests. +This includes KVM_IRQ_ROUTING_XEN_EVTCHN as well. diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 888598fdf543..11b9ff2bd901 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3009,6 +3009,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_PIT_STATE2: case KVM_CAP_SET_IDENTITY_MAP_ADDR: case KVM_CAP_XEN_HVM: + case KVM_CAP_XEN_HVM_GUEST: case KVM_CAP_VCPU_EVENTS: case KVM_CAP_HYPERV: case KVM_CAP_HYPERV_VAPIC: diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index b91e57d9e6d3..682ea00abd58 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1003,6 +1003,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_ARM_VM_IPA_SIZE 165 #define KVM_CAP_MANUAL_DIRTY_LOG_PROTECT 166 #define KVM_CAP_HYPERV_CPUID 167 +#define KVM_CAP_XEN_HVM_GUEST 168 #ifdef KVM_CAP_IRQ_ROUTING @@ -1455,6 +1456,7 @@ struct kvm_enc_region { /* Available with KVM_CAP_HYPERV_CPUID */ #define KVM_GET_SUPPORTED_HV_CPUID _IOWR(KVMIO, 0xc1, struct kvm_cpuid2) +/* Available with KVM_CAP_XEN_HVM_GUEST */ #define KVM_XEN_HVM_GET_ATTR _IOWR(KVMIO, 0xc2, struct kvm_xen_hvm_attr) #define KVM_XEN_HVM_SET_ATTR _IOW(KVMIO, 0xc3, struct kvm_xen_hvm_attr) @@ -1472,6 +1474,7 @@ struct kvm_xen_hvm_attr { } u; }; +/* Available with KVM_CAP_XEN_HVM_GUEST */ #define KVM_XEN_ATTR_TYPE_SHARED_INFO 0x0 #define KVM_XEN_ATTR_TYPE_VCPU_INFO 0x1 #define KVM_XEN_ATTR_TYPE_VCPU_TIME_INFO 0x2 From patchwork Wed Feb 20 20:15:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822689 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19943922 for ; Wed, 20 Feb 2019 20:21:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 029812F1C5 for ; Wed, 20 Feb 2019 20:21:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EB26F2F1DC; Wed, 20 Feb 2019 20:21:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 166D22F1CE for ; Wed, 20 Feb 2019 20:21:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727417AbfBTURo (ORCPT ); Wed, 20 Feb 2019 15:17:44 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:52136 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727401AbfBTURn (ORCPT ); Wed, 20 Feb 2019 15:17:43 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8VUv081696; Wed, 20 Feb 2019 20:17:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=UdadlV07KAuI7D1n34KKSaXxz4+/ZDZCYXQmV9TN8mg=; b=kDZrOVFbKcLAgZLCPpS1jLcr26HTjAv0CwoOkdH37dRgjC3cIDQ2IwPyCEPg50Ae6p7p 6PEunlEwz1FeWjy9KG/+tWxL3z8xaDPx8G8N3E3YsmN5TjZpyz0X+jckacLJibo4WYW0 uiroB1EeAPFlqt8jGCWjB+t2ZbzHbhvhSDZQ2tGyygBmrapsKwPbMiquA03VnsJNUwNm 10Xm1HJnjxLn2MqXpxUA9VF58Oy2EqvP9IcMVQfh80GS6eLFCibwV5odgOzciULhmDJA 3tuJyoe0cwjUpTlwxkTuIjxx4MTuNwjx8LH2RS0N1dkgwK2reRc5ei5MtwG732WdxD8u UQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2qp9xu3xfq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:28 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHRBo021624 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:27 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKHPZK015421; Wed, 20 Feb 2019 20:17:27 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:25 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 10/39] KVM: x86/xen: support upcall vector Date: Wed, 20 Feb 2019 20:15:40 +0000 Message-Id: <20190220201609.28290-11-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ankur Arora Add support for HVM_PARAM_CALLBACK_VIA_TYPE_VECTOR and HVM_PARAM_CALLBACK_VIA_TYPE_EVTCHN upcall. Some Xen upcall variants do not have an EOI for received upcalls. We handle that by directly injecting the interrupt in the VMCS instead of going through the LAPIC. Note that the route @vcpu field represents the vcpu index and not a vcpu id. The vcpu_id is architecture specific e.g. on x86 it's set to the apic id by userspace. Co-developed-by: Joao Martins Signed-off-by: Ankur Arora Signed-off-by: Joao Martins --- arch/x86/include/asm/kvm_host.h | 14 ++++++ arch/x86/kvm/irq.c | 14 ++++-- arch/x86/kvm/irq_comm.c | 11 +++++ arch/x86/kvm/xen.c | 106 ++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/xen.h | 9 ++++ include/linux/kvm_host.h | 24 +++++++++ include/uapi/linux/kvm.h | 8 +++ 7 files changed, 183 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 9d388ba0a05c..3305173bf10b 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -534,6 +534,12 @@ struct kvm_vcpu_hv { cpumask_t tlb_flush; }; +struct kvm_xen_callback { + u32 via; + u32 vector; + atomic_t queued; +}; + /* Xen per vcpu emulation context */ struct kvm_vcpu_xen { struct kvm_xen_exit exit; @@ -543,6 +549,7 @@ struct kvm_vcpu_xen { struct pvclock_vcpu_time_info *pv_time; gpa_t steal_time_addr; struct vcpu_runstate_info *steal_time; + struct kvm_xen_callback cb; }; struct kvm_vcpu_arch { @@ -854,6 +861,13 @@ struct kvm_xen { struct shared_info *shinfo; }; +enum kvm_xen_callback_via { + KVM_XEN_CALLBACK_VIA_GSI, + KVM_XEN_CALLBACK_VIA_PCI_INTX, + KVM_XEN_CALLBACK_VIA_VECTOR, + KVM_XEN_CALLBACK_VIA_EVTCHN, +}; + enum kvm_irqchip_mode { KVM_IRQCHIP_NONE, KVM_IRQCHIP_KERNEL, /* created with KVM_CREATE_IRQCHIP */ diff --git a/arch/x86/kvm/irq.c b/arch/x86/kvm/irq.c index faa264822cee..cdb1dbfcc9b1 100644 --- a/arch/x86/kvm/irq.c +++ b/arch/x86/kvm/irq.c @@ -26,6 +26,7 @@ #include "irq.h" #include "i8254.h" #include "x86.h" +#include "xen.h" /* * check if there are pending timer events @@ -61,7 +62,9 @@ static int kvm_cpu_has_extint(struct kvm_vcpu *v) return pending_userspace_extint(v); else return v->kvm->arch.vpic->output; - } else + } else if (kvm_xen_has_interrupt(v) != -1) + return 1; + else return 0; } @@ -119,7 +122,7 @@ int kvm_cpu_has_interrupt(struct kvm_vcpu *v) if (kvm_cpu_has_extint(v)) return 1; - return kvm_apic_has_interrupt(v) != -1; /* LAPIC */ + return kvm_apic_has_interrupt(v) != -1; /* LAPIC */ } EXPORT_SYMBOL_GPL(kvm_cpu_has_interrupt); @@ -135,8 +138,13 @@ static int kvm_cpu_get_extint(struct kvm_vcpu *v) v->arch.pending_external_vector = -1; return vector; - } else + } else { + int vector = kvm_xen_get_interrupt(v); + + if (vector) + return vector; /* Xen */ return kvm_pic_read_irq(v->kvm); /* PIC */ + } } else return -1; } diff --git a/arch/x86/kvm/irq_comm.c b/arch/x86/kvm/irq_comm.c index 3cc3b2d130a0..3b5da18c9ce2 100644 --- a/arch/x86/kvm/irq_comm.c +++ b/arch/x86/kvm/irq_comm.c @@ -36,6 +36,7 @@ #include "lapic.h" #include "hyperv.h" +#include "xen.h" #include "x86.h" static int kvm_set_pic_irq(struct kvm_kernel_irq_routing_entry *e, @@ -176,6 +177,9 @@ int kvm_arch_set_irq_inatomic(struct kvm_kernel_irq_routing_entry *e, int r; switch (e->type) { + case KVM_IRQ_ROUTING_XEN_EVTCHN: + return kvm_xen_set_evtchn(e, kvm, irq_source_id, level, + line_status); case KVM_IRQ_ROUTING_HV_SINT: return kvm_hv_set_sint(e, kvm, irq_source_id, level, line_status); @@ -325,6 +329,13 @@ int kvm_set_routing_entry(struct kvm *kvm, e->hv_sint.vcpu = ue->u.hv_sint.vcpu; e->hv_sint.sint = ue->u.hv_sint.sint; break; + case KVM_IRQ_ROUTING_XEN_EVTCHN: + e->set = kvm_xen_set_evtchn; + e->evtchn.vcpu = ue->u.evtchn.vcpu; + e->evtchn.vector = ue->u.evtchn.vector; + e->evtchn.via = ue->u.evtchn.via; + + return kvm_xen_setup_evtchn(kvm, e); default: return -EINVAL; } diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 4fdc4c71245a..99a3722146d8 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -7,6 +7,7 @@ #include "x86.h" #include "xen.h" +#include "ioapic.h" #include #include @@ -17,6 +18,111 @@ #include "trace.h" +static void *xen_vcpu_info(struct kvm_vcpu *v); + +int kvm_xen_has_interrupt(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(vcpu); + struct vcpu_info *vcpu_info = xen_vcpu_info(vcpu); + + if (!!atomic_read(&vcpu_xen->cb.queued) || (vcpu_info && + test_bit(0, (unsigned long *) &vcpu_info->evtchn_upcall_pending))) + return 1; + + return -1; +} + +int kvm_xen_get_interrupt(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(vcpu); + u32 vector = vcpu_xen->cb.vector; + + if (kvm_xen_has_interrupt(vcpu) == -1) + return 0; + + atomic_set(&vcpu_xen->cb.queued, 0); + return vector; +} + +static int kvm_xen_do_upcall(struct kvm *kvm, u32 dest_vcpu, + u32 via, u32 vector, int level) +{ + struct kvm_vcpu_xen *vcpu_xen; + struct kvm_lapic_irq irq; + struct kvm_vcpu *vcpu; + + if (vector > 0xff || vector < 0x10 || dest_vcpu >= KVM_MAX_VCPUS) + return -EINVAL; + + vcpu = kvm_get_vcpu(kvm, dest_vcpu); + if (!vcpu) + return -EINVAL; + + memset(&irq, 0, sizeof(irq)); + if (via == KVM_XEN_CALLBACK_VIA_VECTOR) { + vcpu_xen = vcpu_to_xen_vcpu(vcpu); + atomic_set(&vcpu_xen->cb.queued, 1); + kvm_make_request(KVM_REQ_EVENT, vcpu); + kvm_vcpu_kick(vcpu); + } else if (via == KVM_XEN_CALLBACK_VIA_EVTCHN) { + irq.shorthand = APIC_DEST_SELF; + irq.dest_mode = APIC_DEST_PHYSICAL; + irq.delivery_mode = APIC_DM_FIXED; + irq.vector = vector; + irq.level = level; + + /* Deliver upcall to a vector on the destination vcpu */ + kvm_irq_delivery_to_apic(kvm, vcpu->arch.apic, &irq, NULL); + } else { + return -EINVAL; + } + + return 0; +} + +int kvm_xen_set_evtchn(struct kvm_kernel_irq_routing_entry *e, + struct kvm *kvm, int irq_source_id, int level, + bool line_status) +{ + /* + * The routing information for the kirq specifies the vector + * on the destination vcpu. + */ + return kvm_xen_do_upcall(kvm, e->evtchn.vcpu, e->evtchn.via, + e->evtchn.vector, level); +} + +int kvm_xen_setup_evtchn(struct kvm *kvm, + struct kvm_kernel_irq_routing_entry *e) +{ + struct kvm_vcpu_xen *vcpu_xen; + struct kvm_vcpu *vcpu = NULL; + + if (e->evtchn.vector > 0xff || e->evtchn.vector < 0x10) + return -EINVAL; + + /* Expect vcpu to be sane */ + if (e->evtchn.vcpu >= KVM_MAX_VCPUS) + return -EINVAL; + + vcpu = kvm_get_vcpu(kvm, e->evtchn.vcpu); + if (!vcpu) + return -EINVAL; + + vcpu_xen = vcpu_to_xen_vcpu(vcpu); + if (e->evtchn.via == KVM_XEN_CALLBACK_VIA_VECTOR) { + vcpu_xen->cb.via = KVM_XEN_CALLBACK_VIA_VECTOR; + vcpu_xen->cb.vector = e->evtchn.vector; + } else if (e->evtchn.via == KVM_XEN_CALLBACK_VIA_EVTCHN) { + vcpu_xen->cb.via = KVM_XEN_CALLBACK_VIA_EVTCHN; + vcpu_xen->cb.vector = e->evtchn.vector; + } else { + return -EINVAL; + } + + return 0; +} + static void set_vcpu_attr(struct kvm_vcpu *v, u16 type, gpa_t gpa, void *addr) { struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(v); diff --git a/arch/x86/kvm/xen.h b/arch/x86/kvm/xen.h index 2feef68ee80f..6a42e134924a 100644 --- a/arch/x86/kvm/xen.h +++ b/arch/x86/kvm/xen.h @@ -25,6 +25,15 @@ bool kvm_xen_hypercall_enabled(struct kvm *kvm); bool kvm_xen_hypercall_set(struct kvm *kvm); int kvm_xen_hypercall(struct kvm_vcpu *vcpu); +int kvm_xen_has_interrupt(struct kvm_vcpu *vcpu); +int kvm_xen_get_interrupt(struct kvm_vcpu *vcpu); + +int kvm_xen_set_evtchn(struct kvm_kernel_irq_routing_entry *e, + struct kvm *kvm, int irq_source_id, int level, + bool line_status); +int kvm_xen_setup_evtchn(struct kvm *kvm, + struct kvm_kernel_irq_routing_entry *e); + void kvm_xen_destroy_vm(struct kvm *kvm); void kvm_xen_vcpu_uninit(struct kvm_vcpu *vcpu); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 9d55c63db09b..af5e7455ff6a 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -350,6 +350,29 @@ struct kvm_hv_sint { u32 sint; }; +/* + * struct kvm_xen_evtchn: currently specifies the upcall vector setup to + * deliver the interrupt to the guest. + * + * via = XEN_PARAM_CALLBACK_VIA_TYPE_GSI|_PCI + * vcpu: always deliver to vcpu-0 + * vector: is used as upcall-vector + * EOI: none + * via = XEN_PARAM_CALLBACK_VIA_TYPE_VECTOR + * vcpu: deliver to specified vcpu + * vector: used as upcall-vector + * EOI: none + * via = XEN_PARAM_CALLBACK_VIA_TYPE_EVTCHN + * vcpu: deliver to specified vcpu (vector should be bound to the vcpu) + * vector: used as upcall-vector + * EOI: expected + */ +struct kvm_xen_evtchn { + u32 via; + u32 vcpu; + u32 vector; +}; + struct kvm_kernel_irq_routing_entry { u32 gsi; u32 type; @@ -370,6 +393,7 @@ struct kvm_kernel_irq_routing_entry { } msi; struct kvm_s390_adapter_int adapter; struct kvm_hv_sint hv_sint; + struct kvm_xen_evtchn evtchn; }; struct hlist_node link; }; diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 682ea00abd58..49001f681cd1 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1035,11 +1035,18 @@ struct kvm_irq_routing_hv_sint { __u32 sint; }; +struct kvm_irq_routing_xen_evtchn { + __u32 via; + __u32 vcpu; + __u32 vector; +}; + /* gsi routing entry types */ #define KVM_IRQ_ROUTING_IRQCHIP 1 #define KVM_IRQ_ROUTING_MSI 2 #define KVM_IRQ_ROUTING_S390_ADAPTER 3 #define KVM_IRQ_ROUTING_HV_SINT 4 +#define KVM_IRQ_ROUTING_XEN_EVTCHN 5 struct kvm_irq_routing_entry { __u32 gsi; @@ -1051,6 +1058,7 @@ struct kvm_irq_routing_entry { struct kvm_irq_routing_msi msi; struct kvm_irq_routing_s390_adapter adapter; struct kvm_irq_routing_hv_sint hv_sint; + struct kvm_irq_routing_xen_evtchn evtchn; __u32 pad[8]; } u; }; From patchwork Wed Feb 20 20:15:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822685 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3CBCD922 for ; Wed, 20 Feb 2019 20:21:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 29F072F16D for ; Wed, 20 Feb 2019 20:21:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1E6622F194; Wed, 20 Feb 2019 20:21:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3979C2F16D for ; Wed, 20 Feb 2019 20:21:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727545AbfBTURw (ORCPT ); Wed, 20 Feb 2019 15:17:52 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:52224 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727490AbfBTURu (ORCPT ); Wed, 20 Feb 2019 15:17:50 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8ks7081898; Wed, 20 Feb 2019 20:17:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=1Xwv+1Uk4qDQHMQm9A5i5vmdwTT1Em2+yLk6wYsYmxo=; b=yRO5BrCdizWjPJE8uqXqNHPi5A+lCvso+i8HZOJa4MUb0jp/7Oiqs3U79Wm7xoqdU13i g/mOmq/OvPdiZFuMClu4oXXeZxzqD+ZF1x9GseRejA8diJ8mbL4eqfcmUik9i7GMSzeW SnmjyHoIMzMMPLVAitRuxEMzaydBs5OPWODjW09kk1ON5p9EjEBoNPlrEN7tcs4AUe+0 byFjF0RAoYXpBXUTj1ShvE6C2W72T3x9OvPZN0JxGDa00+ygOVuipezokbRbTaotNg6U T7EPCobtnfhZXxTJlv+R2/Oq/pFtUQYZR2e4SN1e41Axc8QZBCmG84K2EZxLXyvRF84s eg== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2130.oracle.com with ESMTP id 2qp9xu3xgb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:35 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHTl9025788 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:30 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHTct011860; Wed, 20 Feb 2019 20:17:29 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:28 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 11/39] KVM: x86/xen: evtchn signaling via eventfd Date: Wed, 20 Feb 2019 20:15:41 +0000 Message-Id: <20190220201609.28290-12-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP userspace registers a @port to an @eventfd, that is bound to a @vcpu. This information is then used when the guest does an EVTCHNOP_send with a port registered with the kernel. EVTCHNOP_send short-circuiting happens by marking the event as pending in the shared info and vcpu info pages and doing the upcall. For IPIs and interdomain event channels, we do the upcall on the assigned vcpu. After binding events the guest or host may wish to bind those events to a particular vcpu. This is usually done for unbound and and interdomain events. Update requests are handled via the KVM_XEN_EVENTFD_UPDATE flag. Unregistered ports are handled by the emulator. Co-developed-by: Ankur Arora Signed-off-by: Joao Martins Signed-off-by: Ankur Arora --- arch/x86/include/asm/kvm_host.h | 3 + arch/x86/kvm/x86.c | 1 + arch/x86/kvm/xen.c | 238 ++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/xen.h | 2 + include/uapi/linux/kvm.h | 20 ++++ 5 files changed, 264 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 3305173bf10b..f31fcaf8fa7c 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -859,6 +859,9 @@ struct kvm_xen { gfn_t shinfo_addr; struct shared_info *shinfo; + + struct idr port_to_evt; + struct mutex xen_lock; }; enum kvm_xen_callback_via { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 11b9ff2bd901..76bd23113ccd 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9181,6 +9181,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) INIT_DELAYED_WORK(&kvm->arch.kvmclock_update_work, kvmclock_update_fn); INIT_DELAYED_WORK(&kvm->arch.kvmclock_sync_work, kvmclock_sync_fn); + kvm_xen_init_vm(kvm); kvm_hv_init_vm(kvm); kvm_page_track_init(kvm); kvm_mmu_init_vm(kvm); diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 99a3722146d8..1fbdfa7c4356 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -10,14 +10,28 @@ #include "ioapic.h" #include +#include #include #include #include #include +#include #include "trace.h" +struct evtchnfd { + struct eventfd_ctx *ctx; + u32 vcpu; + u32 port; + u32 type; + union { + struct { + u8 type; + } virq; + }; +}; + static void *xen_vcpu_info(struct kvm_vcpu *v); int kvm_xen_has_interrupt(struct kvm_vcpu *vcpu) @@ -80,6 +94,13 @@ static int kvm_xen_do_upcall(struct kvm *kvm, u32 dest_vcpu, return 0; } +static void kvm_xen_evtchnfd_upcall(struct kvm_vcpu *vcpu, struct evtchnfd *e) +{ + struct kvm_vcpu_xen *vx = vcpu_to_xen_vcpu(vcpu); + + kvm_xen_do_upcall(vcpu->kvm, e->vcpu, vx->cb.via, vx->cb.vector, 0); +} + int kvm_xen_set_evtchn(struct kvm_kernel_irq_routing_entry *e, struct kvm *kvm, int irq_source_id, int level, bool line_status) @@ -329,6 +350,12 @@ int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) r = 0; break; } + case KVM_XEN_ATTR_TYPE_EVTCHN: { + struct kvm_xen_eventfd xevfd = data->u.evtchn; + + r = kvm_vm_ioctl_xen_eventfd(kvm, &xevfd); + break; + } default: break; } @@ -388,10 +415,96 @@ static int kvm_xen_hypercall_complete_userspace(struct kvm_vcpu *vcpu) return kvm_skip_emulated_instruction(vcpu); } +static int kvm_xen_evtchn_2l_vcpu_set_pending(struct vcpu_info *v) +{ + return test_and_set_bit(0, (unsigned long *) &v->evtchn_upcall_pending); +} + +#define BITS_PER_EVTCHN_WORD (sizeof(xen_ulong_t)*8) + +static int kvm_xen_evtchn_2l_set_pending(struct shared_info *shared_info, + struct vcpu_info *vcpu_info, + int p) +{ + if (test_and_set_bit(p, (unsigned long *) shared_info->evtchn_pending)) + return 1; + + if (!test_bit(p, (unsigned long *) shared_info->evtchn_mask) && + !test_and_set_bit(p / BITS_PER_EVTCHN_WORD, + (unsigned long *) &vcpu_info->evtchn_pending_sel)) + return kvm_xen_evtchn_2l_vcpu_set_pending(vcpu_info); + + return 1; +} + +#undef BITS_PER_EVTCHN_WORD + +static int kvm_xen_evtchn_set_pending(struct kvm_vcpu *svcpu, + struct evtchnfd *evfd) +{ + struct kvm_vcpu_xen *vcpu_xen; + struct vcpu_info *vcpu_info; + struct shared_info *shared_info; + struct kvm_vcpu *vcpu; + + vcpu = kvm_get_vcpu(svcpu->kvm, evfd->vcpu); + if (!vcpu) + return -ENOENT; + + vcpu_xen = vcpu_to_xen_vcpu(vcpu); + shared_info = (struct shared_info *) vcpu->kvm->arch.xen.shinfo; + vcpu_info = (struct vcpu_info *) vcpu_xen->vcpu_info; + + return kvm_xen_evtchn_2l_set_pending(shared_info, vcpu_info, + evfd->port); +} + +static int kvm_xen_evtchn_send(struct kvm_vcpu *vcpu, int port) +{ + struct eventfd_ctx *eventfd; + struct evtchnfd *evtchnfd; + + /* conn_to_evt is protected by vcpu->kvm->srcu */ + evtchnfd = idr_find(&vcpu->kvm->arch.xen.port_to_evt, port); + if (!evtchnfd) + return -ENOENT; + + eventfd = evtchnfd->ctx; + if (!kvm_xen_evtchn_set_pending(vcpu, evtchnfd)) { + if (!eventfd) + kvm_xen_evtchnfd_upcall(vcpu, evtchnfd); + else + eventfd_signal(eventfd, 1); + } + + return 0; +} + +static int kvm_xen_hcall_evtchn_send(struct kvm_vcpu *vcpu, int cmd, u64 param) +{ + struct evtchn_send send; + gpa_t gpa; + int idx; + + /* Port management is done in userspace */ + if (cmd != EVTCHNOP_send) + return -EINVAL; + + idx = srcu_read_lock(&vcpu->kvm->srcu); + gpa = kvm_mmu_gva_to_gpa_system(vcpu, param, NULL); + srcu_read_unlock(&vcpu->kvm->srcu, idx); + + if (!gpa || kvm_vcpu_read_guest(vcpu, gpa, &send, sizeof(send))) + return -EFAULT; + + return kvm_xen_evtchn_send(vcpu, send.port); +} + int kvm_xen_hypercall(struct kvm_vcpu *vcpu) { bool longmode; u64 input, params[5]; + int r; input = (u64)kvm_register_read(vcpu, VCPU_REGS_RAX); @@ -415,6 +528,19 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) trace_kvm_xen_hypercall(input, params[0], params[1], params[2], params[3], params[4]); + switch (input) { + case __HYPERVISOR_event_channel_op: + r = kvm_xen_hcall_evtchn_send(vcpu, params[0], + params[1]); + if (!r) { + kvm_xen_hypercall_set_result(vcpu, r); + return kvm_skip_emulated_instruction(vcpu); + } + /* fallthrough */ + default: + break; + } + vcpu->run->exit_reason = KVM_EXIT_XEN; vcpu->run->xen.type = KVM_EXIT_XEN_HCALL; vcpu->run->xen.u.hcall.input = input; @@ -441,6 +567,12 @@ void kvm_xen_vcpu_uninit(struct kvm_vcpu *vcpu) put_page(virt_to_page(vcpu_xen->steal_time)); } +void kvm_xen_init_vm(struct kvm *kvm) +{ + mutex_init(&kvm->arch.xen.xen_lock); + idr_init(&kvm->arch.xen.port_to_evt); +} + void kvm_xen_destroy_vm(struct kvm *kvm) { struct kvm_xen *xen = &kvm->arch.xen; @@ -448,3 +580,109 @@ void kvm_xen_destroy_vm(struct kvm *kvm) if (xen->shinfo) put_page(virt_to_page(xen->shinfo)); } + +static int kvm_xen_eventfd_update(struct kvm *kvm, struct idr *port_to_evt, + struct mutex *port_lock, + struct kvm_xen_eventfd *args) +{ + struct eventfd_ctx *eventfd = NULL; + struct evtchnfd *evtchnfd; + + mutex_lock(port_lock); + evtchnfd = idr_find(port_to_evt, args->port); + mutex_unlock(port_lock); + + if (!evtchnfd) + return -ENOENT; + + if (args->fd != -1) { + eventfd = eventfd_ctx_fdget(args->fd); + if (IS_ERR(eventfd)) + return PTR_ERR(eventfd); + } + + evtchnfd->vcpu = args->vcpu; + return 0; +} + +static int kvm_xen_eventfd_assign(struct kvm *kvm, struct idr *port_to_evt, + struct mutex *port_lock, + struct kvm_xen_eventfd *args) +{ + struct eventfd_ctx *eventfd = NULL; + struct evtchnfd *evtchnfd; + u32 port = args->port; + int ret; + + if (args->fd != -1) { + eventfd = eventfd_ctx_fdget(args->fd); + if (IS_ERR(eventfd)) + return PTR_ERR(eventfd); + } + + evtchnfd = kzalloc(sizeof(struct evtchnfd), GFP_KERNEL); + if (!evtchnfd) + return -ENOMEM; + + evtchnfd->ctx = eventfd; + evtchnfd->port = port; + evtchnfd->vcpu = args->vcpu; + evtchnfd->type = args->type; + if (evtchnfd->type == XEN_EVTCHN_TYPE_VIRQ) + evtchnfd->virq.type = args->virq.type; + + mutex_lock(port_lock); + ret = idr_alloc(port_to_evt, evtchnfd, port, port + 1, + GFP_KERNEL); + mutex_unlock(port_lock); + + if (ret >= 0) + return 0; + + if (ret == -ENOSPC) + ret = -EEXIST; + + if (eventfd) + eventfd_ctx_put(eventfd); + kfree(evtchnfd); + return ret; +} + +static int kvm_xen_eventfd_deassign(struct kvm *kvm, struct idr *port_to_evt, + struct mutex *port_lock, u32 port) +{ + struct evtchnfd *evtchnfd; + + mutex_lock(port_lock); + evtchnfd = idr_remove(port_to_evt, port); + mutex_unlock(port_lock); + + if (!evtchnfd) + return -ENOENT; + + if (kvm) + synchronize_srcu(&kvm->srcu); + if (evtchnfd->ctx) + eventfd_ctx_put(evtchnfd->ctx); + kfree(evtchnfd); + return 0; +} + +int kvm_vm_ioctl_xen_eventfd(struct kvm *kvm, struct kvm_xen_eventfd *args) +{ + struct kvm_xen *xen = &kvm->arch.xen; + int allowed_flags = (KVM_XEN_EVENTFD_DEASSIGN | KVM_XEN_EVENTFD_UPDATE); + + if ((args->flags & (~allowed_flags)) || + (args->port <= 0)) + return -EINVAL; + + if (args->flags == KVM_XEN_EVENTFD_DEASSIGN) + return kvm_xen_eventfd_deassign(kvm, &xen->port_to_evt, + &xen->xen_lock, args->port); + if (args->flags == KVM_XEN_EVENTFD_UPDATE) + return kvm_xen_eventfd_update(kvm, &xen->port_to_evt, + &xen->xen_lock, args); + return kvm_xen_eventfd_assign(kvm, &xen->port_to_evt, + &xen->xen_lock, args); +} diff --git a/arch/x86/kvm/xen.h b/arch/x86/kvm/xen.h index 6a42e134924a..8f26625564c8 100644 --- a/arch/x86/kvm/xen.h +++ b/arch/x86/kvm/xen.h @@ -34,7 +34,9 @@ int kvm_xen_set_evtchn(struct kvm_kernel_irq_routing_entry *e, int kvm_xen_setup_evtchn(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e); +void kvm_xen_init_vm(struct kvm *kvm); void kvm_xen_destroy_vm(struct kvm *kvm); +int kvm_vm_ioctl_xen_eventfd(struct kvm *kvm, struct kvm_xen_eventfd *args); void kvm_xen_vcpu_uninit(struct kvm_vcpu *vcpu); #endif diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 49001f681cd1..4eae47a0ef63 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1479,6 +1479,25 @@ struct kvm_xen_hvm_attr { __u32 vcpu; __u64 gpa; } vcpu_attr; + struct kvm_xen_eventfd { + +#define XEN_EVTCHN_TYPE_VIRQ 0 +#define XEN_EVTCHN_TYPE_IPI 1 + __u32 type; + __u32 port; + __u32 vcpu; + __s32 fd; + +#define KVM_XEN_EVENTFD_DEASSIGN (1 << 0) +#define KVM_XEN_EVENTFD_UPDATE (1 << 1) + __u32 flags; + union { + struct { + __u8 type; + } virq; + __u32 padding[2]; + }; + } evtchn; } u; }; @@ -1487,6 +1506,7 @@ struct kvm_xen_hvm_attr { #define KVM_XEN_ATTR_TYPE_VCPU_INFO 0x1 #define KVM_XEN_ATTR_TYPE_VCPU_TIME_INFO 0x2 #define KVM_XEN_ATTR_TYPE_VCPU_RUNSTATE 0x3 +#define KVM_XEN_ATTR_TYPE_EVTCHN 0x4 /* Secure Encrypted Virtualization command */ enum sev_cmd_id { From patchwork Wed Feb 20 20:15:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822631 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 285AF13B5 for ; Wed, 20 Feb 2019 20:17:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 154872E97D for ; Wed, 20 Feb 2019 20:17:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 089772EABB; Wed, 20 Feb 2019 20:17:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9FB492E97D for ; Wed, 20 Feb 2019 20:17:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727476AbfBTURs (ORCPT ); Wed, 20 Feb 2019 15:17:48 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:33014 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727401AbfBTURr (ORCPT ); Wed, 20 Feb 2019 15:17:47 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8Vnk087437; Wed, 20 Feb 2019 20:17:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=tmD778ESsfRwNMq9An4G52B5oKCiX42JyVamao6ufmw=; b=TAF37CCyat1YdR1UjRk+dJuoA5zlEnM4afilHLYTgTx8Kc04kxyCiXmxs9Naoerb2cQy uw6u4CmGw4WcJ9VnEG2+v3ir2kF3JSpqKDxxfGPlOo5DB4m9hFDI3n7H+QCvyu3sI4L+ Paz8FjEKZWHzSapv9LDktDuOOUd7KuGvPZwOSehojytBmUYMUodZLdMUz52zt/t23awU nb7FUhixGDSbotApDUPFTNQM/7/vo8LhO+WGserT0vfDgwoHTOLa3WhZOMSRuDu/1ZP2 ohtcPI4ycz6NO6vRQVEPto9xzVF8ecNkSDNJ47EiRrExoAPxBmQY0gGEKJYvDRn1hynz wQ== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2qp81ec39c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:33 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHWjQ025954 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:32 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKHW0X018398; Wed, 20 Feb 2019 20:17:32 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:31 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 12/39] KVM: x86/xen: store virq when assigning evtchn Date: Wed, 20 Feb 2019 20:15:42 +0000 Message-Id: <20190220201609.28290-13-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=941 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Enable virq offload to the hypervisor. The primary user for this is the timer virq. Signed-off-by: Joao Martins --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/xen.c | 23 ++++++++++++++++++++++- 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index f31fcaf8fa7c..92b76127eb43 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -550,6 +550,8 @@ struct kvm_vcpu_xen { gpa_t steal_time_addr; struct vcpu_runstate_info *steal_time; struct kvm_xen_callback cb; +#define KVM_XEN_NR_VIRQS 24 + unsigned int virq_to_port[KVM_XEN_NR_VIRQS]; }; struct kvm_vcpu_arch { diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 1fbdfa7c4356..42c1fe01600d 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -101,6 +101,20 @@ static void kvm_xen_evtchnfd_upcall(struct kvm_vcpu *vcpu, struct evtchnfd *e) kvm_xen_do_upcall(vcpu->kvm, e->vcpu, vx->cb.via, vx->cb.vector, 0); } +void kvm_xen_set_virq(struct kvm *kvm, struct evtchnfd *evt) +{ + int virq = evt->virq.type; + struct kvm_vcpu_xen *vcpu_xen; + struct kvm_vcpu *vcpu; + + vcpu = kvm_get_vcpu(kvm, evt->vcpu); + if (!vcpu) + return; + + vcpu_xen = vcpu_to_xen_vcpu(vcpu); + vcpu_xen->virq_to_port[virq] = evt->port; +} + int kvm_xen_set_evtchn(struct kvm_kernel_irq_routing_entry *e, struct kvm *kvm, int irq_source_id, int level, bool line_status) @@ -620,6 +634,10 @@ static int kvm_xen_eventfd_assign(struct kvm *kvm, struct idr *port_to_evt, return PTR_ERR(eventfd); } + if (args->type == XEN_EVTCHN_TYPE_VIRQ && + args->virq.type >= KVM_XEN_NR_VIRQS) + return -EINVAL; + evtchnfd = kzalloc(sizeof(struct evtchnfd), GFP_KERNEL); if (!evtchnfd) return -ENOMEM; @@ -636,8 +654,11 @@ static int kvm_xen_eventfd_assign(struct kvm *kvm, struct idr *port_to_evt, GFP_KERNEL); mutex_unlock(port_lock); - if (ret >= 0) + if (ret >= 0) { + if (evtchnfd->type == XEN_EVTCHN_TYPE_VIRQ) + kvm_xen_set_virq(kvm, evtchnfd); return 0; + } if (ret == -ENOSPC) ret = -EEXIST; From patchwork Wed Feb 20 20:15:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822687 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 17868922 for ; Wed, 20 Feb 2019 20:21:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 037F22F0B9 for ; Wed, 20 Feb 2019 20:21:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EBD812F142; Wed, 20 Feb 2019 20:21:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 37AF62F021 for ; Wed, 20 Feb 2019 20:21:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727796AbfBTUVJ (ORCPT ); Wed, 20 Feb 2019 15:21:09 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:35362 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727401AbfBTURv (ORCPT ); Wed, 20 Feb 2019 15:17:51 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8fJJ087627; Wed, 20 Feb 2019 20:17:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=WA5tVmPGiyTbR+w8aie+g3q7493Oumr1VfMgf/HFYL4=; b=hgZxKqZ3ohnMpGS0b2ik2rt1//i4DoVo9WbHgxahyBRo4Ios5OHO5eGO9a6CtcxmHDWA 1qaxnTOWFw1+YRrLvQFlkO81CjI5W+Wfl8waR6UCogQbqPeZdw35pGC8hzEhfbNuCsVx yPcTcx+Afa6GJxW3wqmU+tTkNXdY3nybW+FUyLxXyL5ISQHuidBb0c9v+4GurGONV7as yjOg4AfvNq3CE1Dp25HNSoEoXC1n7227sA3pyiIOGdmv0t1jS6vreQ60W+0V54O/coql PfpAVIUgNCasblXo3g+Vswuvp1EFP7RVBCfyKtfxeFxsOhAIYI8hhmQvi3pjxv0Z3iHd kQ== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2qpb5rktqg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:36 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHZJv026107 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:36 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHZXl009502; Wed, 20 Feb 2019 20:17:35 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:35 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 13/39] KVM: x86/xen: handle PV timers oneshot mode Date: Wed, 20 Feb 2019 20:15:43 +0000 Message-Id: <20190220201609.28290-14-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If the guest has offloaded the timer virq, handle the following hypercalls for programming the timer: VCPUOP_set_singleshot_timer VCPUOP_stop_singleshot_timer set_timer_op(timestamp_ns) The event channel corresponding to the timer virq is then used to inject events once timer deadlines are met. For now we back the PV timer with hrtimer. Signed-off-by: Joao Martins --- arch/x86/include/asm/kvm_host.h | 2 + arch/x86/kvm/irq.c | 11 ++- arch/x86/kvm/x86.c | 4 + arch/x86/kvm/xen.c | 185 +++++++++++++++++++++++++++++++++++++++- arch/x86/kvm/xen.h | 6 ++ 5 files changed, 202 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 92b76127eb43..7fcc81dbb688 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -552,6 +552,8 @@ struct kvm_vcpu_xen { struct kvm_xen_callback cb; #define KVM_XEN_NR_VIRQS 24 unsigned int virq_to_port[KVM_XEN_NR_VIRQS]; + struct hrtimer timer; + atomic_t timer_pending; }; struct kvm_vcpu_arch { diff --git a/arch/x86/kvm/irq.c b/arch/x86/kvm/irq.c index cdb1dbfcc9b1..936c31ae019a 100644 --- a/arch/x86/kvm/irq.c +++ b/arch/x86/kvm/irq.c @@ -34,10 +34,14 @@ */ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu) { + int r = 0; + if (lapic_in_kernel(vcpu)) - return apic_has_pending_timer(vcpu); + r = apic_has_pending_timer(vcpu); + if (kvm_xen_timer_enabled(vcpu)) + r += kvm_xen_has_pending_timer(vcpu); - return 0; + return r; } EXPORT_SYMBOL(kvm_cpu_has_pending_timer); @@ -172,6 +176,8 @@ void kvm_inject_pending_timer_irqs(struct kvm_vcpu *vcpu) { if (lapic_in_kernel(vcpu)) kvm_inject_apic_timer_irqs(vcpu); + if (kvm_xen_timer_enabled(vcpu)) + kvm_xen_inject_timer_irqs(vcpu); } EXPORT_SYMBOL_GPL(kvm_inject_pending_timer_irqs); @@ -179,4 +185,5 @@ void __kvm_migrate_timers(struct kvm_vcpu *vcpu) { __kvm_migrate_apic_timer(vcpu); __kvm_migrate_pit_timer(vcpu); + __kvm_migrate_xen_timer(vcpu); } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 76bd23113ccd..e29cefd2dc6a 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9115,6 +9115,7 @@ int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu) vcpu->arch.preempted_in_kernel = false; kvm_hv_vcpu_init(vcpu); + kvm_xen_vcpu_init(vcpu); return 0; @@ -9566,6 +9567,9 @@ static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu) if (kvm_hv_has_stimer_pending(vcpu)) return true; + if (kvm_xen_has_pending_timer(vcpu)) + return true; + return false; } diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 42c1fe01600d..ec40cb1de6b6 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -32,6 +32,7 @@ struct evtchnfd { }; }; +static int kvm_xen_evtchn_send(struct kvm_vcpu *vcpu, int port); static void *xen_vcpu_info(struct kvm_vcpu *v); int kvm_xen_has_interrupt(struct kvm_vcpu *vcpu) @@ -101,6 +102,91 @@ static void kvm_xen_evtchnfd_upcall(struct kvm_vcpu *vcpu, struct evtchnfd *e) kvm_xen_do_upcall(vcpu->kvm, e->vcpu, vx->cb.via, vx->cb.vector, 0); } +int kvm_xen_has_pending_timer(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(vcpu); + + if (kvm_xen_hypercall_enabled(vcpu->kvm) && kvm_xen_timer_enabled(vcpu)) + return atomic_read(&vcpu_xen->timer_pending); + + return 0; +} + +void kvm_xen_inject_timer_irqs(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(vcpu); + + if (atomic_read(&vcpu_xen->timer_pending) > 0) { + kvm_xen_evtchn_send(vcpu, vcpu_xen->virq_to_port[VIRQ_TIMER]); + + atomic_set(&vcpu_xen->timer_pending, 0); + } +} + +static enum hrtimer_restart xen_timer_callback(struct hrtimer *timer) +{ + struct kvm_vcpu_xen *vcpu_xen = + container_of(timer, struct kvm_vcpu_xen, timer); + struct kvm_vcpu *vcpu = xen_vcpu_to_vcpu(vcpu_xen); + struct swait_queue_head *wq = &vcpu->wq; + + if (atomic_read(&vcpu_xen->timer_pending)) + return HRTIMER_NORESTART; + + atomic_inc(&vcpu_xen->timer_pending); + kvm_set_pending_timer(vcpu); + + if (swait_active(wq)) + swake_up_one(wq); + + return HRTIMER_NORESTART; +} + +void __kvm_migrate_xen_timer(struct kvm_vcpu *vcpu) +{ + struct hrtimer *timer; + + if (!kvm_xen_timer_enabled(vcpu)) + return; + + timer = &vcpu->arch.xen.timer; + if (hrtimer_cancel(timer)) + hrtimer_start_expires(timer, HRTIMER_MODE_ABS_PINNED); +} + +static void kvm_xen_start_timer(struct kvm_vcpu *vcpu, u64 delta_ns) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(vcpu); + struct hrtimer *timer = &vcpu_xen->timer; + ktime_t ktime_now; + + atomic_set(&vcpu_xen->timer_pending, 0); + ktime_now = ktime_get(); + hrtimer_start(timer, ktime_add_ns(ktime_now, delta_ns), + HRTIMER_MODE_ABS_PINNED); +} + +static void kvm_xen_stop_timer(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(vcpu); + + hrtimer_cancel(&vcpu_xen->timer); +} + +void kvm_xen_init_timer(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(vcpu); + + hrtimer_init(&vcpu_xen->timer, CLOCK_MONOTONIC, + HRTIMER_MODE_ABS_PINNED); + vcpu_xen->timer.function = xen_timer_callback; +} + +bool kvm_xen_timer_enabled(struct kvm_vcpu *vcpu) +{ + return !!vcpu->arch.xen.virq_to_port[VIRQ_TIMER]; +} + void kvm_xen_set_virq(struct kvm *kvm, struct evtchnfd *evt) { int virq = evt->virq.type; @@ -111,6 +197,9 @@ void kvm_xen_set_virq(struct kvm *kvm, struct evtchnfd *evt) if (!vcpu) return; + if (virq == VIRQ_TIMER) + kvm_xen_init_timer(vcpu); + vcpu_xen = vcpu_to_xen_vcpu(vcpu); vcpu_xen->virq_to_port[virq] = evt->port; } @@ -514,6 +603,71 @@ static int kvm_xen_hcall_evtchn_send(struct kvm_vcpu *vcpu, int cmd, u64 param) return kvm_xen_evtchn_send(vcpu, send.port); } +static int kvm_xen_hcall_vcpu_op(struct kvm_vcpu *vcpu, int cmd, int vcpu_id, + u64 param) +{ + struct vcpu_set_singleshot_timer oneshot; + int ret = -EINVAL; + long delta; + gpa_t gpa; + int idx; + + /* Only process timer ops with commands 6 to 9 */ + if (cmd < VCPUOP_set_periodic_timer || + cmd > VCPUOP_stop_singleshot_timer) + return ret; + + if (!kvm_xen_timer_enabled(vcpu)) + return ret; + + idx = srcu_read_lock(&vcpu->kvm->srcu); + gpa = kvm_mmu_gva_to_gpa_system(vcpu, param, NULL); + srcu_read_unlock(&vcpu->kvm->srcu, idx); + + if (!gpa) + return ret; + + switch (cmd) { + case VCPUOP_set_singleshot_timer: + if (kvm_vcpu_read_guest(vcpu, gpa, &oneshot, + sizeof(oneshot))) + return -EFAULT; + + delta = oneshot.timeout_abs_ns - get_kvmclock_ns(vcpu->kvm); + kvm_xen_start_timer(vcpu, delta); + ret = 0; + break; + case VCPUOP_stop_singleshot_timer: + kvm_xen_stop_timer(vcpu); + ret = 0; + break; + default: + break; + } + + return ret; +} + +static int kvm_xen_hcall_set_timer_op(struct kvm_vcpu *vcpu, uint64_t timeout) +{ + ktime_t ktime_now = ktime_get(); + long delta = timeout - get_kvmclock_ns(vcpu->kvm); + + if (!kvm_xen_timer_enabled(vcpu)) + return -EINVAL; + + if (timeout == 0) { + kvm_xen_stop_timer(vcpu); + } else if (unlikely(timeout < ktime_now) || + ((uint32_t) (delta >> 50) != 0)) { + kvm_xen_start_timer(vcpu, 50000000); + } else { + kvm_xen_start_timer(vcpu, delta); + } + + return 0; +} + int kvm_xen_hypercall(struct kvm_vcpu *vcpu) { bool longmode; @@ -546,10 +700,20 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) case __HYPERVISOR_event_channel_op: r = kvm_xen_hcall_evtchn_send(vcpu, params[0], params[1]); - if (!r) { - kvm_xen_hypercall_set_result(vcpu, r); - return kvm_skip_emulated_instruction(vcpu); - } + if (!r) + goto hcall_success; + break; + case __HYPERVISOR_vcpu_op: + r = kvm_xen_hcall_vcpu_op(vcpu, params[0], params[1], + params[2]); + if (!r) + goto hcall_success; + break; + case __HYPERVISOR_set_timer_op: + r = kvm_xen_hcall_set_timer_op(vcpu, params[0]); + if (!r) + goto hcall_success; + break; /* fallthrough */ default: break; @@ -567,6 +731,14 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) kvm_xen_hypercall_complete_userspace; return 0; + +hcall_success: + kvm_xen_hypercall_set_result(vcpu, r); + return kvm_skip_emulated_instruction(vcpu); +} + +void kvm_xen_vcpu_init(struct kvm_vcpu *vcpu) +{ } void kvm_xen_vcpu_uninit(struct kvm_vcpu *vcpu) @@ -579,6 +751,11 @@ void kvm_xen_vcpu_uninit(struct kvm_vcpu *vcpu) put_page(virt_to_page(vcpu_xen->pv_time)); if (vcpu_xen->steal_time) put_page(virt_to_page(vcpu_xen->steal_time)); + + if (!kvm_xen_timer_enabled(vcpu)) + return; + + kvm_xen_stop_timer(vcpu); } void kvm_xen_init_vm(struct kvm *kvm) diff --git a/arch/x86/kvm/xen.h b/arch/x86/kvm/xen.h index 8f26625564c8..f82b8b5b3345 100644 --- a/arch/x86/kvm/xen.h +++ b/arch/x86/kvm/xen.h @@ -37,6 +37,12 @@ int kvm_xen_setup_evtchn(struct kvm *kvm, void kvm_xen_init_vm(struct kvm *kvm); void kvm_xen_destroy_vm(struct kvm *kvm); int kvm_vm_ioctl_xen_eventfd(struct kvm *kvm, struct kvm_xen_eventfd *args); +void kvm_xen_vcpu_init(struct kvm_vcpu *vcpu); void kvm_xen_vcpu_uninit(struct kvm_vcpu *vcpu); +void __kvm_migrate_xen_timer(struct kvm_vcpu *vcpu); +int kvm_xen_has_pending_timer(struct kvm_vcpu *vcpu); +void kvm_xen_inject_timer_irqs(struct kvm_vcpu *vcpu); +bool kvm_xen_timer_enabled(struct kvm_vcpu *vcpu); + #endif From patchwork Wed Feb 20 20:15:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822633 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 35B291399 for ; Wed, 20 Feb 2019 20:17:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 224E82E97D for ; Wed, 20 Feb 2019 20:17:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 131E52EABB; Wed, 20 Feb 2019 20:17:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AE8452E97D for ; Wed, 20 Feb 2019 20:17:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727604AbfBTURz (ORCPT ); Wed, 20 Feb 2019 15:17:55 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:33106 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727538AbfBTURw (ORCPT ); Wed, 20 Feb 2019 15:17:52 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8e45087617; Wed, 20 Feb 2019 20:17:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=y2ywQsg8ZwocwXwGKOAVPtXD+yYTYrFhFyWzqli6irA=; b=vip4g3s1j01PcCB7aJEcvW5bIh5NTRelBrf8d5VELn1ejI5Hf4lxTVMQ5C7dlGwvIgRY 2ULqlnlcLUfcvqgKW1UKnwd9GDrUnLL+8nDtnPAIANF3f8RmMOHKMYRNM3Ptc9yiIvb7 ESA3TZ9X7DVPwXQil9Inx1WRBjFT5ZEFluxhlTIQcLsL8xpPa0ux7pbwg5c4tv7og+tN N+IN05uHAjz6nQ6TjwgA3m6aE5K5RNQfeObGMyN3JN/YW/FpUPan2b9EBpX1hrCv6EZu gMNkwuk3TJrBheZk7qM1ovZSBRlUelyzjFwSxhX7IO/Sy5lnB9cL/aQEgWsmVaOu0/j8 lw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2qp81ec39n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:39 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHcjF022141 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:38 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKHc8q018497; Wed, 20 Feb 2019 20:17:38 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:38 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 14/39] KVM: x86/xen: handle PV IPI vcpu yield Date: Wed, 20 Feb 2019 20:15:44 +0000 Message-Id: <20190220201609.28290-15-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=907 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Cooperative Linux guests after an IPI-many may yield vcpu if any of the IPI'd vcpus were preempted (i.e. runstate is 'runnable'.) Support SCHEDOP_yield for handling yield. Signed-off-by: Joao Martins --- arch/x86/kvm/xen.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index ec40cb1de6b6..753a6d2c11cd 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -17,6 +17,7 @@ #include #include #include +#include #include "trace.h" @@ -668,6 +669,31 @@ static int kvm_xen_hcall_set_timer_op(struct kvm_vcpu *vcpu, uint64_t timeout) return 0; } +static int kvm_xen_hcall_sched_op(struct kvm_vcpu *vcpu, int cmd, u64 param) +{ + int ret = -ENOSYS; + gpa_t gpa; + int idx; + + idx = srcu_read_lock(&vcpu->kvm->srcu); + gpa = kvm_mmu_gva_to_gpa_system(vcpu, param, NULL); + srcu_read_unlock(&vcpu->kvm->srcu, idx); + + if (!gpa) + return -EFAULT; + + switch (cmd) { + case SCHEDOP_yield: + kvm_vcpu_on_spin(vcpu, true); + ret = 0; + break; + default: + break; + } + + return ret; +} + int kvm_xen_hypercall(struct kvm_vcpu *vcpu) { bool longmode; @@ -714,6 +740,11 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) if (!r) goto hcall_success; break; + case __HYPERVISOR_sched_op: + r = kvm_xen_hcall_sched_op(vcpu, params[0], params[1]); + if (!r) + goto hcall_success; + break; /* fallthrough */ default: break; From patchwork Wed Feb 20 20:15:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822635 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 56AD413B5 for ; Wed, 20 Feb 2019 20:18:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 363332E97D for ; Wed, 20 Feb 2019 20:18:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 271832EABE; Wed, 20 Feb 2019 20:18:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7B01D2E97D for ; Wed, 20 Feb 2019 20:18:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727673AbfBTUR7 (ORCPT ); Wed, 20 Feb 2019 15:17:59 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:35478 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727629AbfBTUR6 (ORCPT ); Wed, 20 Feb 2019 15:17:58 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8Z4o087545; Wed, 20 Feb 2019 20:17:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=qBbl27sHElfrv/UD8a84o+I89etq4k6ZdourMYL0T0Y=; b=IY8CS4+RKNYXD7dKDvyKMn+88JSyOhD/QgBU8Bf5BI4MINsPZw86zGoj9hN2cd1QUoEZ 18c8DYOjkeyO5NFzPIv0cIJTaGdtgB7pOGFutf/YhMiqtpp6QQnMErJcNCekYK55cgIP ZbcKMIyge8RuxKX/Qa27Orv8BHep+S/ht0XSPhhZ2foQ/hklZJRXRjn1TPBs7dS/ZmPo XVoMc5j1dqn2P71vv92AXYBWFgwbFhEUXXETOx+NDJtJ5zBSjoRdBSYMzu1Jel7W+2LG vwVvd9LJJO4jIQsilV5wiW28oZs4a+85Lx33fgT+LgfrfSQqVBaZ3HeONB1QL2za6Iu7 Ag== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2qpb5rktqx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:43 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHg3F004724 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:42 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHfUr011919; Wed, 20 Feb 2019 20:17:41 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:41 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 15/39] KVM: x86/xen: handle PV spinlocks slowpath Date: Wed, 20 Feb 2019 20:15:45 +0000 Message-Id: <20190220201609.28290-16-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Boris Ostrovsky Add support for SCHEDOP_poll hypercall. This implementation is optimized for polling for a single channel, which is what Linux does. Polling for multiple channels is not especially efficient (and has not been tested). PV spinlocks slow path uses this hypercall, and explicitly crash if it's not supported. Signed-off-by: Boris Ostrovsky --- arch/x86/include/asm/kvm_host.h | 3 ++ arch/x86/kvm/xen.c | 108 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 111 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 7fcc81dbb688..c629fedb2e21 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -554,6 +554,8 @@ struct kvm_vcpu_xen { unsigned int virq_to_port[KVM_XEN_NR_VIRQS]; struct hrtimer timer; atomic_t timer_pending; + wait_queue_head_t sched_waitq; + int poll_evtchn; }; struct kvm_vcpu_arch { @@ -865,6 +867,7 @@ struct kvm_xen { struct shared_info *shinfo; struct idr port_to_evt; + unsigned long poll_mask[BITS_TO_LONGS(KVM_MAX_VCPUS)]; struct mutex xen_lock; }; diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 753a6d2c11cd..07066402737d 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -563,6 +563,16 @@ static int kvm_xen_evtchn_set_pending(struct kvm_vcpu *svcpu, evfd->port); } +static void kvm_xen_check_poller(struct kvm_vcpu *vcpu, int port) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(vcpu); + + if ((vcpu_xen->poll_evtchn == port || + vcpu_xen->poll_evtchn == -1) && + test_and_clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.xen.poll_mask)) + wake_up(&vcpu_xen->sched_waitq); +} + static int kvm_xen_evtchn_send(struct kvm_vcpu *vcpu, int port) { struct eventfd_ctx *eventfd; @@ -581,6 +591,8 @@ static int kvm_xen_evtchn_send(struct kvm_vcpu *vcpu, int port) eventfd_signal(eventfd, 1); } + kvm_xen_check_poller(kvm_get_vcpu(vcpu->kvm, evtchnfd->vcpu), port); + return 0; } @@ -669,6 +681,94 @@ static int kvm_xen_hcall_set_timer_op(struct kvm_vcpu *vcpu, uint64_t timeout) return 0; } +static bool wait_pending_event(struct kvm_vcpu *vcpu, int nr_ports, + evtchn_port_t *ports) +{ + int i; + struct shared_info *shared_info = + (struct shared_info *)vcpu->kvm->arch.xen.shinfo; + + for (i = 0; i < nr_ports; i++) + if (test_bit(ports[i], + (unsigned long *)shared_info->evtchn_pending)) + return true; + + return false; +} + +static int kvm_xen_schedop_poll(struct kvm_vcpu *vcpu, gpa_t gpa) +{ + struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(vcpu); + int idx, i; + struct sched_poll sched_poll; + evtchn_port_t port, *ports; + struct shared_info *shared_info; + struct evtchnfd *evtchnfd; + int ret = 0; + + if (kvm_vcpu_read_guest(vcpu, gpa, + &sched_poll, sizeof(sched_poll))) + return -EFAULT; + + shared_info = (struct shared_info *)vcpu->kvm->arch.xen.shinfo; + + if (unlikely(sched_poll.nr_ports > 1)) { + /* Xen (unofficially) limits number of pollers to 128 */ + if (sched_poll.nr_ports > 128) + return -EINVAL; + + ports = kmalloc_array(sched_poll.nr_ports, + sizeof(*ports), GFP_KERNEL); + if (!ports) + return -ENOMEM; + } else + ports = &port; + + set_bit(vcpu->vcpu_id, vcpu->kvm->arch.xen.poll_mask); + + for (i = 0; i < sched_poll.nr_ports; i++) { + idx = srcu_read_lock(&vcpu->kvm->srcu); + gpa = kvm_mmu_gva_to_gpa_system(vcpu, + (gva_t)(sched_poll.ports + i), + NULL); + srcu_read_unlock(&vcpu->kvm->srcu, idx); + + if (!gpa || kvm_vcpu_read_guest(vcpu, gpa, + &ports[i], sizeof(port))) { + ret = -EFAULT; + goto out; + } + + evtchnfd = idr_find(&vcpu->kvm->arch.xen.port_to_evt, + ports[i]); + if (!evtchnfd) { + ret = -ENOENT; + goto out; + } + } + + if (sched_poll.nr_ports == 1) + vcpu_xen->poll_evtchn = port; + else + vcpu_xen->poll_evtchn = -1; + + if (!wait_pending_event(vcpu, sched_poll.nr_ports, ports)) + wait_event_interruptible_timeout( + vcpu_xen->sched_waitq, + wait_pending_event(vcpu, sched_poll.nr_ports, ports), + sched_poll.timeout ?: KTIME_MAX); + + vcpu_xen->poll_evtchn = 0; + +out: + /* Really, this is only needed in case of timeout */ + clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.xen.poll_mask); + + if (unlikely(sched_poll.nr_ports > 1)) + kfree(ports); + return ret; +} + static int kvm_xen_hcall_sched_op(struct kvm_vcpu *vcpu, int cmd, u64 param) { int ret = -ENOSYS; @@ -687,6 +787,9 @@ static int kvm_xen_hcall_sched_op(struct kvm_vcpu *vcpu, int cmd, u64 param) kvm_vcpu_on_spin(vcpu, true); ret = 0; break; + case SCHEDOP_poll: + ret = kvm_xen_schedop_poll(vcpu, gpa); + break; default: break; } @@ -744,6 +847,9 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) r = kvm_xen_hcall_sched_op(vcpu, params[0], params[1]); if (!r) goto hcall_success; + else if (params[0] == SCHEDOP_poll) + /* SCHEDOP_poll should be handled in kernel */ + return r; break; /* fallthrough */ default: @@ -770,6 +876,8 @@ int kvm_xen_hypercall(struct kvm_vcpu *vcpu) void kvm_xen_vcpu_init(struct kvm_vcpu *vcpu) { + init_waitqueue_head(&vcpu->arch.xen.sched_waitq); + vcpu->arch.xen.poll_evtchn = 0; } void kvm_xen_vcpu_uninit(struct kvm_vcpu *vcpu) From patchwork Wed Feb 20 20:15:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822637 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0D84213B5 for ; Wed, 20 Feb 2019 20:18:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EF45D2E97D for ; Wed, 20 Feb 2019 20:18:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E2E6F2EABE; Wed, 20 Feb 2019 20:18:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 85B052E97D for ; Wed, 20 Feb 2019 20:18:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727718AbfBTUSF (ORCPT ); Wed, 20 Feb 2019 15:18:05 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:33222 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727660AbfBTUR7 (ORCPT ); Wed, 20 Feb 2019 15:17:59 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8VDJ087434; Wed, 20 Feb 2019 20:17:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=iMKfAofvTA0lJaBbYGohbYbz6+eKYt5uJa2e5hpVk2Q=; b=T6t+61v9WlponIdHDeeGEW/SrwSmAPkFXqwbI1ktwHI/k/jCuwnN7Nfmg7njzwLedDRt geW003NOn/utXdzsUgbVOeodnYHFi8qzBXwnQ9OpolOkAH2A3IaQ84Yw4H7hKmekVnQB tkWabO4WpJ8SRNxwVYK/SejHAVjRg0ka98MM7YNwZSPOCTbdVAetf4MeE/vjDsZ9Rn36 TxZ3ox4z3InKAzwzlaVzvnRjOS9emxupfRsNwyl0hDdeMEitJqhGW0eOA/Xuw8KRQMvU i8sE9Oww+fPQuzXZIEELOypKtD1nOP36WhfwY4MOO7aUpktmPASrq/RC8KRzu7J//Zv6 0A== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2qp81ec3a7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:45 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHikT022432 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:45 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHia2009656; Wed, 20 Feb 2019 20:17:44 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:44 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 16/39] KVM: x86: declare Xen HVM evtchn offload capability Date: Wed, 20 Feb 2019 20:15:46 +0000 Message-Id: <20190220201609.28290-17-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=726 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add new capability for event channel support in hypervisor. Signed-off-by: Joao Martins --- Documentation/virtual/kvm/api.txt | 9 +++++++++ arch/x86/kvm/x86.c | 1 + include/uapi/linux/kvm.h | 3 +++ 3 files changed, 13 insertions(+) diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt index c3a1402b76bf..36d9386415fa 100644 --- a/Documentation/virtual/kvm/api.txt +++ b/Documentation/virtual/kvm/api.txt @@ -5037,3 +5037,12 @@ Architectures: x86 This capability indicates that KVM supports Xen HVM guests. This includes KVM_IRQ_ROUTING_XEN_EVTCHN as well. + +8.23 KVM_CAP_XEN_HVM_EVTCHN + +Architectures: x86 + +This capability indicates KVM's support for the event channel offload. +Implies support for KVM_IRQ_ROUTING_XEN_EVTCHN irq routing, and +for attribute KVM_XEN_ATTR_TYPE_EVTCHN in KVM_XEN_HVM_GET_ATTR or +KVM_XEN_HVM_SET_ATTR. diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index e29cefd2dc6a..b1d9045d7989 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3010,6 +3010,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_SET_IDENTITY_MAP_ADDR: case KVM_CAP_XEN_HVM: case KVM_CAP_XEN_HVM_GUEST: + case KVM_CAP_XEN_HVM_EVTCHN: case KVM_CAP_VCPU_EVENTS: case KVM_CAP_HYPERV: case KVM_CAP_HYPERV_VAPIC: diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 4eae47a0ef63..1b3ecce5f92e 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1004,6 +1004,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_MANUAL_DIRTY_LOG_PROTECT 166 #define KVM_CAP_HYPERV_CPUID 167 #define KVM_CAP_XEN_HVM_GUEST 168 +#define KVM_CAP_XEN_HVM_EVTCHN 169 #ifdef KVM_CAP_IRQ_ROUTING @@ -1046,6 +1047,7 @@ struct kvm_irq_routing_xen_evtchn { #define KVM_IRQ_ROUTING_MSI 2 #define KVM_IRQ_ROUTING_S390_ADAPTER 3 #define KVM_IRQ_ROUTING_HV_SINT 4 +/* Available with KVM_CAP_XEN_HVM_EVTCHN */ #define KVM_IRQ_ROUTING_XEN_EVTCHN 5 struct kvm_irq_routing_entry { @@ -1506,6 +1508,7 @@ struct kvm_xen_hvm_attr { #define KVM_XEN_ATTR_TYPE_VCPU_INFO 0x1 #define KVM_XEN_ATTR_TYPE_VCPU_TIME_INFO 0x2 #define KVM_XEN_ATTR_TYPE_VCPU_RUNSTATE 0x3 +/* Available with KVM_CAP_XEN_HVM_EVTCHN */ #define KVM_XEN_ATTR_TYPE_EVTCHN 0x4 /* Secure Encrypted Virtualization command */ From patchwork Wed Feb 20 20:15:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822657 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 689231805 for ; Wed, 20 Feb 2019 20:19:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 54FD32F01F for ; Wed, 20 Feb 2019 20:19:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 467F42F021; Wed, 20 Feb 2019 20:19:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA02A2F01F for ; Wed, 20 Feb 2019 20:19:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728190AbfBTUTM (ORCPT ); Wed, 20 Feb 2019 15:19:12 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:53548 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727119AbfBTUTI (ORCPT ); Wed, 20 Feb 2019 15:19:08 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8Uhd081679; Wed, 20 Feb 2019 20:17:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=sI+XoHhsjwJV0n1ws79+QJSD983LQFkAU2y/PjLPk54=; b=eLwhTvA43e+K0d0bPGEgP41YXPJQzaqcvn6fgjcmYtzoZjPU/Jq31SFsUnjnNQTgEk57 LKOqVxnhnuhsI3vz63C4fVzC5HY9sIEVb5P/3dzU+l6IQt48dDWBnX5wFoegnPprbPIt NnY1EieQX9sBJGIqJ3dGmFlp5mG0BZaJ7+XERbimD1MirCWA1Qef3YoWZfl1l5nNI6pa PucYr7X3xpx7JKU0HTO5M3rBBj6Z6JlX76XCxVgjvlx1XCtUsAyN29Zu+6KombBFZoIw ejxluXLvEvvR8b0zoaMRAkupHOV56iD0wItkkAQyhcNlDCDlExUgSl+5rlavefa+6cxb Xg== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2130.oracle.com with ESMTP id 2qp9xu3xhn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:49 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHnb5026797 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:49 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKHmCQ015556; Wed, 20 Feb 2019 20:17:48 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:48 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org, Juergen Gross , Stefano Stabellini Subject: [PATCH RFC 17/39] x86/xen: export vcpu_info and shared_info Date: Wed, 20 Feb 2019 20:15:47 +0000 Message-Id: <20190220201609.28290-18-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ankur Arora Also remove __init annotations from xen_evtchn_2l/fifo_init(). This allows us to support 2-level event channel ABI on xen_shim_domain(). Signed-off-by: Ankur Arora --- arch/x86/xen/enlighten.c | 3 +++ drivers/xen/events/events_2l.c | 2 +- drivers/xen/events/events_fifo.c | 2 +- include/xen/xen-ops.h | 2 +- 4 files changed, 6 insertions(+), 3 deletions(-) diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index 750f46ad018a..73b9736e89d2 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -50,6 +50,8 @@ DEFINE_PER_CPU(struct vcpu_info, xen_vcpu_info); /* Linux <-> Xen vCPU id mapping */ DEFINE_PER_CPU(uint32_t, xen_vcpu_id); EXPORT_PER_CPU_SYMBOL(xen_vcpu_id); +EXPORT_PER_CPU_SYMBOL(xen_vcpu); +EXPORT_PER_CPU_SYMBOL(xen_vcpu_info); enum xen_domain_type xen_domain_type = XEN_NATIVE; EXPORT_SYMBOL_GPL(xen_domain_type); @@ -79,6 +81,7 @@ EXPORT_SYMBOL(xen_start_flags); * page as soon as fixmap is up and running. */ struct shared_info *HYPERVISOR_shared_info = &xen_dummy_shared_info; +EXPORT_SYMBOL_GPL(HYPERVISOR_shared_info); /* * Flag to determine whether vcpu info placement is available on all diff --git a/drivers/xen/events/events_2l.c b/drivers/xen/events/events_2l.c index 8edef51c92e5..b5acf4b09971 100644 --- a/drivers/xen/events/events_2l.c +++ b/drivers/xen/events/events_2l.c @@ -369,7 +369,7 @@ static const struct evtchn_ops evtchn_ops_2l = { .resume = evtchn_2l_resume, }; -void __init xen_evtchn_2l_init(void) +void xen_evtchn_2l_init(void) { pr_info("Using 2-level ABI\n"); evtchn_ops = &evtchn_ops_2l; diff --git a/drivers/xen/events/events_fifo.c b/drivers/xen/events/events_fifo.c index 76b318e88382..453b4b05f238 100644 --- a/drivers/xen/events/events_fifo.c +++ b/drivers/xen/events/events_fifo.c @@ -430,7 +430,7 @@ static int xen_evtchn_cpu_dead(unsigned int cpu) return 0; } -int __init xen_evtchn_fifo_init(void) +int xen_evtchn_fifo_init(void) { int cpu = smp_processor_id(); int ret; diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h index 4969817124a8..92fc45075500 100644 --- a/include/xen/xen-ops.h +++ b/include/xen/xen-ops.h @@ -10,7 +10,7 @@ #include DECLARE_PER_CPU(struct vcpu_info *, xen_vcpu); - +DECLARE_PER_CPU(struct vcpu_info, xen_vcpu_info); DECLARE_PER_CPU(uint32_t, xen_vcpu_id); static inline uint32_t xen_vcpu_nr(int cpu) { From patchwork Wed Feb 20 20:15:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822641 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 94A2713B5 for ; Wed, 20 Feb 2019 20:18:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7C09A2EA35 for ; Wed, 20 Feb 2019 20:18:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 706B82EAD3; Wed, 20 Feb 2019 20:18:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CFEC32EA35 for ; Wed, 20 Feb 2019 20:18:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727701AbfBTUSP (ORCPT ); Wed, 20 Feb 2019 15:18:15 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:35786 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727787AbfBTUSM (ORCPT ); Wed, 20 Feb 2019 15:18:12 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8X06087279; Wed, 20 Feb 2019 20:17:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=yZpRshFok7HHOJ43X643k5AyRcEujMIw9dtV9CXJRJg=; b=1Oozu2huMwf2Gl/VTapxfKIOkytKKYlt4+xaqThxN2OB/hlboNYscxIHX57rQtoWsC8R qQhXT/d8/xX8+cycbr6ou16pSgenmBI198Y5k/5qLwCvdp5yeeb3wPKim9bJr1NCms/R PBDOH54MidArV5/tRW9dnLsONqBwaXxVD+cJy2jQNP6d35XD1SpZPtCRdnIWipGo2rf7 RQQDMwWWYcdcp3Ig4p0QC6nmCa+LyOkBfA0aoLiubxRbYxD4z4OitAD2BypRAmsZkTwM Vph7cApzHUgmuQBwid1qPL6T8fxa9Q8gU54fYFIAa1JW30ZUzL2eCO+Wz6wBpV2iXDdv eQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2qpb5rktrk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:54 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHr3T022699 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:53 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHqvq012033; Wed, 20 Feb 2019 20:17:52 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:51 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org, Juergen Gross , Stefano Stabellini Subject: [PATCH RFC 18/39] x86/xen: make hypercall_page generic Date: Wed, 20 Feb 2019 20:15:48 +0000 Message-Id: <20190220201609.28290-19-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ankur Arora Export hypercall_page as a generic interface which can be implemented by other hypervisors. With this change, hypercall_page now points to the newly introduced xen_hypercall_page which is seeded by Xen, or to one that is filled in by a different hypervisor. Signed-off-by: Ankur Arora --- arch/x86/include/asm/xen/hypercall.h | 12 +++++++----- arch/x86/xen/enlighten.c | 1 + arch/x86/xen/enlighten_hvm.c | 3 ++- arch/x86/xen/enlighten_pv.c | 1 + arch/x86/xen/enlighten_pvh.c | 3 ++- arch/x86/xen/xen-asm_32.S | 2 +- arch/x86/xen/xen-asm_64.S | 2 +- arch/x86/xen/xen-head.S | 8 ++++---- 8 files changed, 19 insertions(+), 13 deletions(-) diff --git a/arch/x86/include/asm/xen/hypercall.h b/arch/x86/include/asm/xen/hypercall.h index ef05bea7010d..1a3cd6680e6f 100644 --- a/arch/x86/include/asm/xen/hypercall.h +++ b/arch/x86/include/asm/xen/hypercall.h @@ -86,11 +86,13 @@ struct xen_dm_op_buf; * there aren't more than 5 arguments...) */ -extern struct { char _entry[32]; } hypercall_page[]; +struct hypercall_entry { char _entry[32]; }; +extern struct hypercall_entry xen_hypercall_page[128]; +extern struct hypercall_entry *hypercall_page; -#define __HYPERCALL "call hypercall_page+%c[offset]" +#define __HYPERCALL CALL_NOSPEC #define __HYPERCALL_ENTRY(x) \ - [offset] "i" (__HYPERVISOR_##x * sizeof(hypercall_page[0])) + [thunk_target] "0" (hypercall_page + __HYPERVISOR_##x) #ifdef CONFIG_X86_32 #define __HYPERCALL_RETREG "eax" @@ -116,7 +118,7 @@ extern struct { char _entry[32]; } hypercall_page[]; register unsigned long __arg4 asm(__HYPERCALL_ARG4REG) = __arg4; \ register unsigned long __arg5 asm(__HYPERCALL_ARG5REG) = __arg5; -#define __HYPERCALL_0PARAM "=r" (__res), ASM_CALL_CONSTRAINT +#define __HYPERCALL_0PARAM "=&r" (__res), ASM_CALL_CONSTRAINT #define __HYPERCALL_1PARAM __HYPERCALL_0PARAM, "+r" (__arg1) #define __HYPERCALL_2PARAM __HYPERCALL_1PARAM, "+r" (__arg2) #define __HYPERCALL_3PARAM __HYPERCALL_2PARAM, "+r" (__arg3) @@ -208,7 +210,7 @@ xen_single_call(unsigned int call, asm volatile(CALL_NOSPEC : __HYPERCALL_5PARAM - : [thunk_target] "a" (&hypercall_page[call]) + : [thunk_target] "0" (hypercall_page + call) : __HYPERCALL_CLOBBER5); return (long)__res; diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index 73b9736e89d2..b36a10e6b5d7 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -20,6 +20,7 @@ #include "smp.h" #include "pmu.h" +struct hypercall_entry *hypercall_page; EXPORT_SYMBOL_GPL(hypercall_page); /* diff --git a/arch/x86/xen/enlighten_hvm.c b/arch/x86/xen/enlighten_hvm.c index 0e75642d42a3..40845e3e9a96 100644 --- a/arch/x86/xen/enlighten_hvm.c +++ b/arch/x86/xen/enlighten_hvm.c @@ -105,8 +105,9 @@ static void __init init_hvm_pv_info(void) pv_info.name = "Xen HVM"; msr = cpuid_ebx(base + 2); - pfn = __pa(hypercall_page); + pfn = __pa(xen_hypercall_page); wrmsr_safe(msr, (u32)pfn, (u32)(pfn >> 32)); + hypercall_page = xen_hypercall_page; } xen_setup_features(); diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c index c54a493e139a..e1537713b57d 100644 --- a/arch/x86/xen/enlighten_pv.c +++ b/arch/x86/xen/enlighten_pv.c @@ -1197,6 +1197,7 @@ asmlinkage __visible void __init xen_start_kernel(void) if (!xen_start_info) return; + hypercall_page = xen_hypercall_page; xen_domain_type = XEN_PV_DOMAIN; xen_start_flags = xen_start_info->flags; diff --git a/arch/x86/xen/enlighten_pvh.c b/arch/x86/xen/enlighten_pvh.c index 35b7599d2d0b..d57a8ad1769e 100644 --- a/arch/x86/xen/enlighten_pvh.c +++ b/arch/x86/xen/enlighten_pvh.c @@ -30,8 +30,9 @@ void __init xen_pvh_init(void) xen_start_flags = pvh_start_info.flags; msr = cpuid_ebx(xen_cpuid_base() + 2); - pfn = __pa(hypercall_page); + pfn = __pa(xen_hypercall_page); wrmsr_safe(msr, (u32)pfn, (u32)(pfn >> 32)); + hypercall_page = xen_hypercall_page; } void __init mem_map_via_hcall(struct boot_params *boot_params_p) diff --git a/arch/x86/xen/xen-asm_32.S b/arch/x86/xen/xen-asm_32.S index c15db060a242..ee4998055ea9 100644 --- a/arch/x86/xen/xen-asm_32.S +++ b/arch/x86/xen/xen-asm_32.S @@ -121,7 +121,7 @@ xen_iret_end_crit: hyper_iret: /* put this out of line since its very rarely used */ - jmp hypercall_page + __HYPERVISOR_iret * 32 + jmp xen_hypercall_page + __HYPERVISOR_iret * 32 .globl xen_iret_start_crit, xen_iret_end_crit diff --git a/arch/x86/xen/xen-asm_64.S b/arch/x86/xen/xen-asm_64.S index 1e9ef0ba30a5..2172d6aec9a3 100644 --- a/arch/x86/xen/xen-asm_64.S +++ b/arch/x86/xen/xen-asm_64.S @@ -70,7 +70,7 @@ ENTRY(xen_early_idt_handler_array) END(xen_early_idt_handler_array) __FINIT -hypercall_iret = hypercall_page + __HYPERVISOR_iret * 32 +hypercall_iret = xen_hypercall_page + __HYPERVISOR_iret * 32 /* * Xen64 iret frame: * diff --git a/arch/x86/xen/xen-head.S b/arch/x86/xen/xen-head.S index 5077ead5e59c..7ff5437bd83f 100644 --- a/arch/x86/xen/xen-head.S +++ b/arch/x86/xen/xen-head.S @@ -58,18 +58,18 @@ END(startup_xen) .pushsection .text .balign PAGE_SIZE -ENTRY(hypercall_page) +ENTRY(xen_hypercall_page) .rept (PAGE_SIZE / 32) UNWIND_HINT_EMPTY .skip 32 .endr #define HYPERCALL(n) \ - .equ xen_hypercall_##n, hypercall_page + __HYPERVISOR_##n * 32; \ + .equ xen_hypercall_##n, xen_hypercall_page + __HYPERVISOR_##n * 32; \ .type xen_hypercall_##n, @function; .size xen_hypercall_##n, 32 #include #undef HYPERCALL -END(hypercall_page) +END(xen_hypercall_page) .popsection ELFNOTE(Xen, XEN_ELFNOTE_GUEST_OS, .asciz "linux") @@ -85,7 +85,7 @@ END(hypercall_page) #ifdef CONFIG_XEN_PV ELFNOTE(Xen, XEN_ELFNOTE_ENTRY, _ASM_PTR startup_xen) #endif - ELFNOTE(Xen, XEN_ELFNOTE_HYPERCALL_PAGE, _ASM_PTR hypercall_page) + ELFNOTE(Xen, XEN_ELFNOTE_HYPERCALL_PAGE, _ASM_PTR xen_hypercall_page) ELFNOTE(Xen, XEN_ELFNOTE_FEATURES, .ascii "!writable_page_tables|pae_pgdir_above_4gb") ELFNOTE(Xen, XEN_ELFNOTE_SUPPORTED_FEATURES, From patchwork Wed Feb 20 20:15:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822639 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5F9FC1399 for ; Wed, 20 Feb 2019 20:18:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4BE0F2E97D for ; Wed, 20 Feb 2019 20:18:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3FE522EABE; Wed, 20 Feb 2019 20:18:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AECF12E97D for ; Wed, 20 Feb 2019 20:18:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727754AbfBTUSK (ORCPT ); Wed, 20 Feb 2019 15:18:10 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:33442 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727622AbfBTUSJ (ORCPT ); Wed, 20 Feb 2019 15:18:09 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8VnE087431; Wed, 20 Feb 2019 20:18:02 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=Q7ek5QGqUVAsNrW8yhTeyoULt5uKOYECFtZLQ9l/1ho=; b=fkNNkExhVsWOCn+cW2Vbb1Z1v8D4qYjbf8P+te0V44dWN4Y0hLDXs1/d3mLjVYtofmmD CQjjw4g6xYExMumH484CoTDUMSomlsUplIYUFUDafOCSKlu1R+VgNdczG+BBw7bmlqAB L1ilXPc6z91hd9Li9rE3PCIAgQwLl1OKuTroxfrUNdaMW6eKDwgk2lQ/dJ5fDzEPt55M uuQ2eT29Gsy0yTF0rlJO0coPgCSTAW2mVetS7UCnbdAWHLqvaRLMss143y0cLQ9pPHXU /7EN3cUIIe6jn3Pbe/Rl99WP74EnCFT+riUfEuKhpXIRA94lxJAlkhfXWI6RoVnIZhQu JA== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2qp81ec3by-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:01 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHtvi027093 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:55 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKHtjM015587; Wed, 20 Feb 2019 20:17:55 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:54 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Juergen Gross , Stefano Stabellini Subject: [PATCH RFC 19/39] xen/xenbus: xenbus uninit support Date: Wed, 20 Feb 2019 20:15:49 +0000 Message-Id: <20190220201609.28290-20-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This allows reinitialization of xenbus which is useful for xen_shim_domain() support. Cleaning xenbus state means cancelling pending watch events, and deleting all watches, closing xenstore event channel and finally stopping xenbus/xenwatch kthreads alongside unregistering /proc/xen. Signed-off-by: Joao Martins --- drivers/xen/xenbus/xenbus.h | 2 ++ drivers/xen/xenbus/xenbus_client.c | 5 ++++ drivers/xen/xenbus/xenbus_probe.c | 51 +++++++++++++++++++++++++++++++++++--- drivers/xen/xenbus/xenbus_xs.c | 38 ++++++++++++++++++++++++++++ 4 files changed, 93 insertions(+), 3 deletions(-) diff --git a/drivers/xen/xenbus/xenbus.h b/drivers/xen/xenbus/xenbus.h index 092981171df1..e0e586d81d48 100644 --- a/drivers/xen/xenbus/xenbus.h +++ b/drivers/xen/xenbus/xenbus.h @@ -96,6 +96,7 @@ extern wait_queue_head_t xb_waitq; extern struct mutex xb_write_mutex; int xs_init(void); +void xs_deinit(void); int xb_init_comms(void); void xb_deinit_comms(void); int xs_watch_msg(struct xs_watch_event *event); @@ -129,6 +130,7 @@ int xenbus_read_otherend_details(struct xenbus_device *xendev, char *id_node, char *path_node); void xenbus_ring_ops_init(void); +void xenbus_ring_ops_deinit(void); int xenbus_dev_request_and_reply(struct xsd_sockmsg *msg, void *par); void xenbus_dev_queue_reply(struct xb_req_data *req); diff --git a/drivers/xen/xenbus/xenbus_client.c b/drivers/xen/xenbus/xenbus_client.c index e17ca8156171..ada1c9aa6525 100644 --- a/drivers/xen/xenbus/xenbus_client.c +++ b/drivers/xen/xenbus/xenbus_client.c @@ -935,3 +935,8 @@ void __init xenbus_ring_ops_init(void) #endif ring_ops = &ring_ops_hvm; } + +void xenbus_ring_ops_deinit(void) +{ + ring_ops = NULL; +} diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index 5b471889d723..2e0ed46b05e7 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -741,6 +741,21 @@ static int __init xenstored_local_init(void) return err; } +static void xenstored_local_deinit(void) +{ + struct evtchn_close close; + void *page = NULL; + + page = gfn_to_virt(xen_store_gfn); + free_page((unsigned long)page); + + close.port = xen_store_evtchn; + + HYPERVISOR_event_channel_op(EVTCHNOP_close, &close); + + xen_store_evtchn = 0; +} + static int xenbus_resume_cb(struct notifier_block *nb, unsigned long action, void *data) { @@ -765,7 +780,11 @@ static struct notifier_block xenbus_resume_nb = { .notifier_call = xenbus_resume_cb, }; -static int __init xenbus_init(void) +#ifdef CONFIG_XEN_COMPAT_XENFS +struct proc_dir_entry *xen_procfs; +#endif + +int xenbus_init(void) { int err = 0; uint64_t v = 0; @@ -833,13 +852,39 @@ static int __init xenbus_init(void) * Create xenfs mountpoint in /proc for compatibility with * utilities that expect to find "xenbus" under "/proc/xen". */ - proc_create_mount_point("xen"); + xen_procfs = proc_create_mount_point("xen"); #endif out_error: return err; } - +EXPORT_SYMBOL_GPL(xenbus_init); postcore_initcall(xenbus_init); +void xenbus_deinit(void) +{ + if (!xen_domain()) + return; + +#ifdef CONFIG_XEN_COMPAT_XENFS + proc_remove(xen_procfs); + xen_procfs = NULL; +#endif + + xs_deinit(); + xenstored_ready = 0; + + switch (xen_store_domain_type) { + case XS_LOCAL: + xenstored_local_deinit(); + xen_store_interface = NULL; + break; + default: + pr_warn("Xenstore state unknown\n"); + break; + } + xenbus_ring_ops_deinit(); +} +EXPORT_SYMBOL_GPL(xenbus_deinit); + MODULE_LICENSE("GPL"); diff --git a/drivers/xen/xenbus/xenbus_xs.c b/drivers/xen/xenbus/xenbus_xs.c index 49a3874ae6bb..bd6db3703972 100644 --- a/drivers/xen/xenbus/xenbus_xs.c +++ b/drivers/xen/xenbus/xenbus_xs.c @@ -866,6 +866,7 @@ static int xenwatch_thread(void *unused) for (;;) { wait_event_interruptible(watch_events_waitq, + kthread_should_stop() || !list_empty(&watch_events)); if (kthread_should_stop()) @@ -917,6 +918,8 @@ static struct notifier_block xs_reboot_nb = { .notifier_call = xs_reboot_notify, }; +static struct task_struct *xenwatch_task; + int xs_init(void) { int err; @@ -932,9 +935,44 @@ int xs_init(void) task = kthread_run(xenwatch_thread, NULL, "xenwatch"); if (IS_ERR(task)) return PTR_ERR(task); + xenwatch_task = task; /* shutdown watches for kexec boot */ xs_reset_watches(); return 0; } + +void cancel_watches(void) +{ + struct xs_watch_event *event, *tmp; + + /* Cancel pending watch events. */ + spin_lock(&watch_events_lock); + list_for_each_entry_safe(event, tmp, &watch_events, list) { + list_del(&event->list); + kfree(event); + } + spin_unlock(&watch_events_lock); +} + +void delete_watches(void) +{ + struct xenbus_watch *watch, *tmp; + + spin_lock(&watches_lock); + list_for_each_entry_safe(watch, tmp, &watches, list) { + list_del(&watch->list); + } + spin_unlock(&watches_lock); +} + +void xs_deinit(void) +{ + kthread_stop(xenwatch_task); + xenwatch_task = NULL; + xb_deinit_comms(); + unregister_reboot_notifier(&xs_reboot_nb); + cancel_watches(); + delete_watches(); +} From patchwork Wed Feb 20 20:15:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822659 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 91113922 for ; Wed, 20 Feb 2019 20:19:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7E7B22F01F for ; Wed, 20 Feb 2019 20:19:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 71A412F021; Wed, 20 Feb 2019 20:19:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 11EA52F01F for ; Wed, 20 Feb 2019 20:19:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728139AbfBTUTE (ORCPT ); Wed, 20 Feb 2019 15:19:04 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:36708 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727119AbfBTUTE (ORCPT ); Wed, 20 Feb 2019 15:19:04 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8c9Z087605; Wed, 20 Feb 2019 20:17:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=77QyT0phQkeZUA6ZmD10vXuqjVPqFaqksd3QgbzVgcE=; b=VpdkMv1bV3DD6gUFdqPFCR+s/5qiMvSTU6egNXo5eskWdCjlpH7Pc3p1u9v9BTn4Dvyj GS3WMCVZfIa/AgRBQWC/eBtNEbZEQsbwEY60yBa+KGQNpxBFKfyIBRdNqZA5jjtL0nOK +zo5l0AEagcNHPQ/Y5JCWMJpoal74XhAZTmo3MggSrSMajzZiwKmTBBhcltMWkWtn6/U Yfpojqn8WBblNy/veaQq/QflH0sVXLDoKD+rST7fLIXTTsJwPpjvGb78CQ3ZFi1qSlBk iubArrqmcyp4l55QLarPbme+MgDFypFru3cnoBO7mMwa454c6EEJXDuTx3VBSsOoWiDk kw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2qpb5rktsd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:59 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHwrh022956 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:17:58 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKHvwI012065; Wed, 20 Feb 2019 20:17:57 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:17:57 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Konrad Rzeszutek Wilk , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= Subject: [PATCH RFC 20/39] xen-blkback: module_exit support Date: Wed, 20 Feb 2019 20:15:50 +0000 Message-Id: <20190220201609.28290-21-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Implement module_exit to allow users to do module unload of blkback. We prevent users from module unload whenever there are still interfaces allocated, in other words, do module_get on xen_blkif_alloc() and module_put on xen_blkif_free(). Signed-off-by: Joao Martins --- drivers/block/xen-blkback/blkback.c | 8 ++++++++ drivers/block/xen-blkback/common.h | 2 ++ drivers/block/xen-blkback/xenbus.c | 14 ++++++++++++++ 3 files changed, 24 insertions(+) diff --git a/drivers/block/xen-blkback/blkback.c b/drivers/block/xen-blkback/blkback.c index fd1e19f1a49f..d51d88be88e1 100644 --- a/drivers/block/xen-blkback/blkback.c +++ b/drivers/block/xen-blkback/blkback.c @@ -1504,5 +1504,13 @@ static int __init xen_blkif_init(void) module_init(xen_blkif_init); +static void __exit xen_blkif_exit(void) +{ + xen_blkif_interface_exit(); + xen_blkif_xenbus_exit(); +} + +module_exit(xen_blkif_exit); + MODULE_LICENSE("Dual BSD/GPL"); MODULE_ALIAS("xen-backend:vbd"); diff --git a/drivers/block/xen-blkback/common.h b/drivers/block/xen-blkback/common.h index 1d3002d773f7..3415c558e115 100644 --- a/drivers/block/xen-blkback/common.h +++ b/drivers/block/xen-blkback/common.h @@ -376,8 +376,10 @@ struct phys_req { blkif_sector_t sector_number; }; int xen_blkif_interface_init(void); +void xen_blkif_interface_exit(void); int xen_blkif_xenbus_init(void); +void xen_blkif_xenbus_exit(void); irqreturn_t xen_blkif_be_int(int irq, void *dev_id); int xen_blkif_schedule(void *arg); diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c index a4bc74e72c39..424e2efebe85 100644 --- a/drivers/block/xen-blkback/xenbus.c +++ b/drivers/block/xen-blkback/xenbus.c @@ -181,6 +181,8 @@ static struct xen_blkif *xen_blkif_alloc(domid_t domid) init_completion(&blkif->drain_complete); INIT_WORK(&blkif->free_work, xen_blkif_deferred_free); + __module_get(THIS_MODULE); + return blkif; } @@ -328,6 +330,8 @@ static void xen_blkif_free(struct xen_blkif *blkif) /* Make sure everything is drained before shutting down */ kmem_cache_free(xen_blkif_cachep, blkif); + + module_put(THIS_MODULE); } int __init xen_blkif_interface_init(void) @@ -341,6 +345,11 @@ int __init xen_blkif_interface_init(void) return 0; } +void xen_blkif_interface_exit(void) +{ + kmem_cache_destroy(xen_blkif_cachep); +} + /* * sysfs interface for VBD I/O requests */ @@ -1115,3 +1124,8 @@ int xen_blkif_xenbus_init(void) { return xenbus_register_backend(&xen_blkbk_driver); } + +void xen_blkif_xenbus_exit(void) +{ + xenbus_unregister_driver(&xen_blkbk_driver); +} From patchwork Wed Feb 20 20:15:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822683 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 694941399 for ; Wed, 20 Feb 2019 20:21:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 568FF2F05B for ; Wed, 20 Feb 2019 20:21:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4A9242F198; Wed, 20 Feb 2019 20:21:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B7C7D2F152 for ; Wed, 20 Feb 2019 20:20:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727837AbfBTUSW (ORCPT ); Wed, 20 Feb 2019 15:18:22 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:33644 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727813AbfBTUSV (ORCPT ); Wed, 20 Feb 2019 15:18:21 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8Y1B087453; Wed, 20 Feb 2019 20:18:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=Ocu54pk3NtEKCI8wPFFHUf3OhFUmnOlU+mn28ZBsLxs=; b=eyDCvbxcxlnQUnAhXMWTq12RS2kENa36wky7g0bR1uaDpPp0n12Qhvrbw8nqE73ssvaN gS9dCp2T+M5HyF/lEevsnAGGLa0bzokAJSf4pXTnSUmLwewez/yjWISzLmKl3NTElg36 Sp3G9VGWANNuTWww/+KtnC4ouasEH6SM3DGDdkLgbItPQ6lm7jrcbY+eapIHTUUiexgx iygX6DtGHYMnBmmwJlzYnCkEsRGnhU8f/2kcGSj22Cvvcm6ilvRqWoCppiue7M3Dp6eU pQ4m6z1r87WWBFyDsewGFF+4Wxov7EEp1WG6b9jv83U3PHWmCdlwLMVBi7uK8pmViWhz gA== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2qp81ec3cn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:07 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKI0fu027446 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:01 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKI0x3009753; Wed, 20 Feb 2019 20:18:00 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:00 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 21/39] KVM: x86/xen: domid allocation Date: Wed, 20 Feb 2019 20:15:51 +0000 Message-Id: <20190220201609.28290-22-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=636 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Userspace requests a free @domid to be assigned to itself, or explicitly selects one by setting @any to 0. The @domid is then used for various interdomain/unbound event purposes. Signed-off-by: Joao Martins --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/x86.c | 2 ++ arch/x86/kvm/xen.c | 70 +++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/xen.h | 2 ++ include/uapi/linux/kvm.h | 4 +++ 5 files changed, 80 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index c629fedb2e21..384247fc433d 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -27,6 +27,7 @@ #include #include #include +#include #include #include @@ -862,6 +863,7 @@ struct kvm_hv { /* Xen emulation context */ struct kvm_xen { u64 xen_hypercall; + domid_t domid; gfn_t shinfo_addr; struct shared_info *shinfo; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b1d9045d7989..cb95f7f8bed9 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -6986,6 +6986,7 @@ int kvm_arch_init(void *opaque) if (hypervisor_is_type(X86_HYPER_MS_HYPERV)) set_hv_tscchange_cb(kvm_hyperv_tsc_notifier); #endif + kvm_xen_init(); return 0; @@ -6999,6 +7000,7 @@ int kvm_arch_init(void *opaque) void kvm_arch_exit(void) { + kvm_xen_exit(); #ifdef CONFIG_X86_64 if (hypervisor_is_type(X86_HYPER_MS_HYPERV)) clear_hv_tscchange_cb(); diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 07066402737d..e570c9b26563 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -36,6 +36,48 @@ struct evtchnfd { static int kvm_xen_evtchn_send(struct kvm_vcpu *vcpu, int port); static void *xen_vcpu_info(struct kvm_vcpu *v); +#define XEN_DOMID_MIN 1 +#define XEN_DOMID_MAX (DOMID_FIRST_RESERVED - 1) + +static rwlock_t domid_lock; +static struct idr domid_to_kvm; + +static int kvm_xen_domid_init(struct kvm *kvm, bool any, domid_t domid) +{ + u16 min = XEN_DOMID_MIN, max = XEN_DOMID_MAX; + struct kvm_xen *xen = &kvm->arch.xen; + int ret; + + if (!any) { + min = domid; + max = domid + 1; + } + + write_lock_bh(&domid_lock); + ret = idr_alloc(&domid_to_kvm, kvm, min, max, GFP_ATOMIC); + write_unlock_bh(&domid_lock); + + if (ret < 0) + return ret; + + xen->domid = ret; + return 0; +} + +int kvm_xen_free_domid(struct kvm *kvm) +{ + struct kvm_xen *xen = &kvm->arch.xen; + struct kvm *vm; + + write_lock_bh(&domid_lock); + vm = idr_remove(&domid_to_kvm, xen->domid); + write_unlock_bh(&domid_lock); + + synchronize_srcu(&kvm->srcu); + + return vm == kvm; +} + int kvm_xen_has_interrupt(struct kvm_vcpu *vcpu) { struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(vcpu); @@ -460,6 +502,17 @@ int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) r = kvm_vm_ioctl_xen_eventfd(kvm, &xevfd); break; } + case KVM_XEN_ATTR_TYPE_DOMID: { + domid_t domid = (u16) data->u.dom.domid; + bool any = (data->u.dom.domid < 0); + + /* Domain ID 0 or >= 0x7ff0 are reserved */ + if (!any && (!domid || (domid >= XEN_DOMID_MAX))) + return -EINVAL; + + r = kvm_xen_domid_init(kvm, any, domid); + break; + } default: break; } @@ -489,6 +542,11 @@ int kvm_xen_hvm_get_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) r = 0; break; } + case KVM_XEN_ATTR_TYPE_DOMID: { + data->u.dom.domid = kvm->arch.xen.domid; + r = 0; + break; + } default: break; } @@ -909,6 +967,18 @@ void kvm_xen_destroy_vm(struct kvm *kvm) if (xen->shinfo) put_page(virt_to_page(xen->shinfo)); + + kvm_xen_free_domid(kvm); +} + +void kvm_xen_init(void) +{ + idr_init(&domid_to_kvm); + rwlock_init(&domid_lock); +} + +void kvm_xen_exit(void) +{ } static int kvm_xen_eventfd_update(struct kvm *kvm, struct idr *port_to_evt, diff --git a/arch/x86/kvm/xen.h b/arch/x86/kvm/xen.h index f82b8b5b3345..76ef2150c650 100644 --- a/arch/x86/kvm/xen.h +++ b/arch/x86/kvm/xen.h @@ -39,6 +39,8 @@ void kvm_xen_destroy_vm(struct kvm *kvm); int kvm_vm_ioctl_xen_eventfd(struct kvm *kvm, struct kvm_xen_eventfd *args); void kvm_xen_vcpu_init(struct kvm_vcpu *vcpu); void kvm_xen_vcpu_uninit(struct kvm_vcpu *vcpu); +void kvm_xen_init(void); +void kvm_xen_exit(void); void __kvm_migrate_xen_timer(struct kvm_vcpu *vcpu); int kvm_xen_has_pending_timer(struct kvm_vcpu *vcpu); diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 1b3ecce5f92e..3212cad732dd 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1500,6 +1500,9 @@ struct kvm_xen_hvm_attr { __u32 padding[2]; }; } evtchn; + struct { + __s32 domid; + } dom; } u; }; @@ -1510,6 +1513,7 @@ struct kvm_xen_hvm_attr { #define KVM_XEN_ATTR_TYPE_VCPU_RUNSTATE 0x3 /* Available with KVM_CAP_XEN_HVM_EVTCHN */ #define KVM_XEN_ATTR_TYPE_EVTCHN 0x4 +#define KVM_XEN_ATTR_TYPE_DOMID 0x5 /* Secure Encrypted Virtualization command */ enum sev_cmd_id { From patchwork Wed Feb 20 20:15:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822643 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 11BE71399 for ; Wed, 20 Feb 2019 20:18:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F2B332E97D for ; Wed, 20 Feb 2019 20:18:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E704A2EABE; Wed, 20 Feb 2019 20:18:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 55AF52E97D for ; Wed, 20 Feb 2019 20:18:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727872AbfBTUS0 (ORCPT ); Wed, 20 Feb 2019 15:18:26 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:36010 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727856AbfBTUSY (ORCPT ); Wed, 20 Feb 2019 15:18:24 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8mh1087654; Wed, 20 Feb 2019 20:18:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=NU46i4dx5S4gBMOqXT0/SvgK/gbayTEmAnpW0YFRcz0=; b=odgzp3Zas0N1pt2dNs73fQjVXeCP72PlzImpjrbGORjlInMYrFmD1azVPFguLdkr1kV2 hYtvLaQHIIjscJs+/uMJ05HVrIu8t9Y/GbuYk0CfqIs5+dEen0kAvXUHpSmnyhde/ZD2 pki7EOXSjZQNcPAnU++7dEzcPs4HcpfYP6XHlPkHATiqMUT4WOfpQW/fqIpGI88OX3BY PwzB8c1r9PwI/iYmGDiCK+wzy7mOCFJR8CzEWue8ZCmrWeXRAwdi9c/BBP7LQjoJKaBc dgMhUY6c7BxV9fDSzSnrVML8ZV/3x5fiyR9gC3ksahm6gTRlv40kLB8b2C6gvrqsqf/S /w== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2qpb5rkttc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:10 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKI4TG006421 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:04 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKI3OM012216; Wed, 20 Feb 2019 20:18:04 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:03 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 22/39] KVM: x86/xen: grant table init Date: Wed, 20 Feb 2019 20:15:52 +0000 Message-Id: <20190220201609.28290-23-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add support for guest grant table initialization. This is mostly scaffolding at this point: we allocate grant table state and map it globally. Later patches add support for seeding the grant table with reserved entries, and setup maptrack which would be used for grant map and unmap operations. Signed-off-by: Joao Martins --- arch/x86/include/asm/kvm_host.h | 19 +++++++++ arch/x86/kvm/xen.c | 88 +++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/xen.h | 1 + include/uapi/linux/kvm.h | 13 ++++++ 4 files changed, 121 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 384247fc433d..e0cbc0899580 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -860,6 +860,23 @@ struct kvm_hv { atomic_t num_mismatched_vp_indexes; }; +/* Xen grant table */ +struct kvm_grant_table { + u32 nr_frames; + u32 max_nr_frames; + union { + void **frames; + struct grant_entry_v1 **frames_v1; + }; + gfn_t *frames_addr; + gpa_t initial_addr; + struct grant_entry_v1 *initial; + + /* maptrack limits */ + u32 max_mt_frames; + u32 nr_mt_frames; +}; + /* Xen emulation context */ struct kvm_xen { u64 xen_hypercall; @@ -871,6 +888,8 @@ struct kvm_xen { struct idr port_to_evt; unsigned long poll_mask[BITS_TO_LONGS(KVM_MAX_VCPUS)]; struct mutex xen_lock; + + struct kvm_grant_table gnttab; }; enum kvm_xen_callback_via { diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index e570c9b26563..b9e6e8f72d87 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include "trace.h" @@ -35,6 +36,7 @@ struct evtchnfd { static int kvm_xen_evtchn_send(struct kvm_vcpu *vcpu, int port); static void *xen_vcpu_info(struct kvm_vcpu *v); +static void kvm_xen_gnttab_free(struct kvm_xen *xen); #define XEN_DOMID_MIN 1 #define XEN_DOMID_MAX (DOMID_FIRST_RESERVED - 1) @@ -513,6 +515,12 @@ int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) r = kvm_xen_domid_init(kvm, any, domid); break; } + case KVM_XEN_ATTR_TYPE_GNTTAB: { + struct kvm_xen_gnttab xevfd = data->u.gnttab; + + r = kvm_vm_ioctl_xen_gnttab(kvm, &xevfd); + break; + } default: break; } @@ -969,6 +977,7 @@ void kvm_xen_destroy_vm(struct kvm *kvm) put_page(virt_to_page(xen->shinfo)); kvm_xen_free_domid(kvm); + kvm_xen_gnttab_free(&kvm->arch.xen); } void kvm_xen_init(void) @@ -1093,3 +1102,82 @@ int kvm_vm_ioctl_xen_eventfd(struct kvm *kvm, struct kvm_xen_eventfd *args) return kvm_xen_eventfd_assign(kvm, &xen->port_to_evt, &xen->xen_lock, args); } + +int kvm_xen_gnttab_init(struct kvm *kvm, struct kvm_xen *xen, + struct kvm_xen_gnttab *op, int dom0) +{ + u32 max_mt_frames = op->init.max_maptrack_frames; + unsigned long initial = op->init.initial_frame; + struct kvm_grant_table *gnttab = &xen->gnttab; + u32 max_frames = op->init.max_frames; + struct page *page = NULL; + void *addr; + + if (!dom0) { + if (!op->init.initial_frame || + offset_in_page(op->init.initial_frame)) + return -EINVAL; + + if (get_user_pages_fast(initial, 1, 1, &page) != 1) + return -EFAULT; + + gnttab->initial_addr = initial; + gnttab->initial = page_to_virt(page); + put_page(page); + } + + addr = kcalloc(max_frames, sizeof(gfn_t), GFP_KERNEL); + if (!addr) + goto out; + xen->gnttab.frames_addr = addr; + + addr = kcalloc(max_frames, sizeof(addr), GFP_KERNEL); + if (!addr) + goto out; + + gnttab->frames = addr; + gnttab->frames[0] = xen->gnttab.initial; + gnttab->max_nr_frames = max_frames; + gnttab->max_mt_frames = max_mt_frames; + gnttab->nr_mt_frames = 1; + gnttab->nr_frames = 0; + + pr_debug("kvm_xen: dom%u: grant table limits (gnttab:%d maptrack:%d)\n", + xen->domid, gnttab->max_nr_frames, gnttab->max_mt_frames); + return 0; + +out: + kfree(xen->gnttab.frames); + kfree(xen->gnttab.frames_addr); + if (page) + put_page(page); + memset(&xen->gnttab, 0, sizeof(xen->gnttab)); + return -ENOMEM; +} + +void kvm_xen_gnttab_free(struct kvm_xen *xen) +{ + struct kvm_grant_table *gnttab = &xen->gnttab; + + kfree(gnttab->frames); + kfree(gnttab->frames_addr); +} + +int kvm_vm_ioctl_xen_gnttab(struct kvm *kvm, struct kvm_xen_gnttab *op) +{ + int r = -EINVAL; + + if (!op) + return r; + + switch (op->flags) { + case KVM_XEN_GNTTAB_F_INIT: + r = kvm_xen_gnttab_init(kvm, &kvm->arch.xen, op, 0); + break; + default: + r = -ENOSYS; + break; + } + + return r; +} diff --git a/arch/x86/kvm/xen.h b/arch/x86/kvm/xen.h index 76ef2150c650..08ad4e1259df 100644 --- a/arch/x86/kvm/xen.h +++ b/arch/x86/kvm/xen.h @@ -37,6 +37,7 @@ int kvm_xen_setup_evtchn(struct kvm *kvm, void kvm_xen_init_vm(struct kvm *kvm); void kvm_xen_destroy_vm(struct kvm *kvm); int kvm_vm_ioctl_xen_eventfd(struct kvm *kvm, struct kvm_xen_eventfd *args); +int kvm_vm_ioctl_xen_gnttab(struct kvm *kvm, struct kvm_xen_gnttab *op); void kvm_xen_vcpu_init(struct kvm_vcpu *vcpu); void kvm_xen_vcpu_uninit(struct kvm_vcpu *vcpu); void kvm_xen_init(void); diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 3212cad732dd..e4fb9bc34d61 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1503,6 +1503,18 @@ struct kvm_xen_hvm_attr { struct { __s32 domid; } dom; + struct kvm_xen_gnttab { +#define KVM_XEN_GNTTAB_F_INIT 0 + __u32 flags; + union { + struct { + __u32 max_frames; + __u32 max_maptrack_frames; + __u64 initial_frame; + } init; + __u32 padding[4]; + }; + } gnttab; } u; }; @@ -1514,6 +1526,7 @@ struct kvm_xen_hvm_attr { /* Available with KVM_CAP_XEN_HVM_EVTCHN */ #define KVM_XEN_ATTR_TYPE_EVTCHN 0x4 #define KVM_XEN_ATTR_TYPE_DOMID 0x5 +#define KVM_XEN_ATTR_TYPE_GNTTAB 0x6 /* Secure Encrypted Virtualization command */ enum sev_cmd_id { From patchwork Wed Feb 20 20:15:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822677 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 79E38922 for ; Wed, 20 Feb 2019 20:20:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 65EFB2F0C6 for ; Wed, 20 Feb 2019 20:20:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 59D252F0B9; Wed, 20 Feb 2019 20:20:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF6D42F0E8 for ; Wed, 20 Feb 2019 20:20:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727924AbfBTUSb (ORCPT ); Wed, 20 Feb 2019 15:18:31 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:33758 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727900AbfBTUS2 (ORCPT ); Wed, 20 Feb 2019 15:18:28 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8Vnp087437; Wed, 20 Feb 2019 20:18:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=7ig7Sg5h1FYfSi3sU0r70ZC4ydxm/2hRabR9MYY/Poo=; b=uOVgl3zk+npTxYojOEiTH2Ghkie7xpuN+jto+FoUtZUfjYXI1v8YxENbYwhKJU2JDUj9 GZmztXGXisR8lJ336quvpcXY2WCmfuAeX16oxAwkUyJQ0Zi3IjFuVyazCYGciBd+mk32 3Y8koDm9j/dduPjxw3xlZ9NzPFl2IALiLO73FGuQODkIz05Hfo1sagEu41eEKOQquVxY QfkGk0THheJNMo5v0a2jSkRFqBNayh4xMcuV9j+flmEDg6vrJ5CC8FzmKeSCj6OVabVr BISlPoJSLCupgyFwfyvX9L10mpbiOZQ7N8XWNeN1GB/QwkUN4sml1d3yQhJ62XyiofM1 pA== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp2130.oracle.com with ESMTP id 2qp81ec3d1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:12 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKI7hm006514 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:07 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKI7Ln009812; Wed, 20 Feb 2019 20:18:07 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:06 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 23/39] KVM: x86/xen: grant table grow support Date: Wed, 20 Feb 2019 20:15:53 +0000 Message-Id: <20190220201609.28290-24-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Guests grant tables with core Xen PV devices (xenbus, console) need to be seeded with a bunch of reserved entries at boot. However, at init, the grant table is, from a guest perspective, empty and has no frames backing it. That only happens once the guest does: XENMEM_add_to_physmap(idx=N,gfn=M,space=XENMAPSPACE_grant_table) Which will share the added page with the hypervisor. The way we handle this then is to seed (from userspace) the initial frame where we store special entries which reference guest PV ring pages. These pages are in-turn mapped/unmapped in backend domains hosting xenstored and xenconsoled. When the guest initializes its grant tables (with the hypercall listed above) we copy the entries from the private frame into a "mapped" gfn. To do this, the userspace VMM handles XENMEM_add_to_physmap hypercall and the hypervisor grows its grant table. Note that a grant table can only grow - no shrinking is possible. Signed-off-by: Joao Martins --- arch/x86/include/asm/kvm_host.h | 16 ++++++++ arch/x86/kvm/xen.c | 90 +++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 5 +++ 3 files changed, 111 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index e0cbc0899580..70bb7339ddd4 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -860,6 +860,21 @@ struct kvm_hv { atomic_t num_mismatched_vp_indexes; }; +struct kvm_grant_map { + u64 gpa; + union { + struct { + +#define _KVM_GNTMAP_ACTIVE (15) +#define KVM_GNTMAP_ACTIVE (1 << _KVM_GNTMAP_ACTIVE) + u16 flags; + u16 ref; + u32 domid; + }; + u64 fields; + }; +}; + /* Xen grant table */ struct kvm_grant_table { u32 nr_frames; @@ -871,6 +886,7 @@ struct kvm_grant_table { gfn_t *frames_addr; gpa_t initial_addr; struct grant_entry_v1 *initial; + struct kvm_grant_map **handle; /* maptrack limits */ u32 max_mt_frames; diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index b9e6e8f72d87..7266d27db210 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -22,6 +22,12 @@ #include "trace.h" +/* Grant v1 references per 4K page */ +#define GPP_V1 (PAGE_SIZE / sizeof(struct grant_entry_v1)) + +/* Grant mappings per 4K page */ +#define MPP (PAGE_SIZE / sizeof(struct kvm_grant_map)) + struct evtchnfd { struct eventfd_ctx *ctx; u32 vcpu; @@ -1158,11 +1164,92 @@ int kvm_xen_gnttab_init(struct kvm *kvm, struct kvm_xen *xen, void kvm_xen_gnttab_free(struct kvm_xen *xen) { struct kvm_grant_table *gnttab = &xen->gnttab; + int i; + + for (i = 0; i < gnttab->nr_frames; i++) + put_page(virt_to_page(gnttab->frames[i])); kfree(gnttab->frames); kfree(gnttab->frames_addr); } +int kvm_xen_gnttab_copy_initial_frame(struct kvm *kvm) +{ + struct kvm_grant_table *gnttab = &kvm->arch.xen.gnttab; + int idx = 0; + + /* Only meant to copy the first gpa being populated */ + if (!gnttab->initial_addr || !gnttab->frames[idx]) + return -EINVAL; + + memcpy(gnttab->frames[idx], gnttab->initial, PAGE_SIZE); + return 0; +} + +int kvm_xen_maptrack_grow(struct kvm_xen *xen, u32 target) +{ + u32 max_entries = target * GPP_V1; + u32 nr_entries = xen->gnttab.nr_mt_frames * MPP; + int i, j, err = 0; + void *addr; + + for (i = nr_entries, j = xen->gnttab.nr_mt_frames; + i < max_entries; i += MPP, j++) { + addr = (void *) get_zeroed_page(GFP_KERNEL); + if (!addr) { + err = -ENOMEM; + break; + } + + xen->gnttab.handle[j] = addr; + } + + xen->gnttab.nr_mt_frames = j; + xen->gnttab.nr_frames = target; + return err; +} + +int kvm_xen_gnttab_grow(struct kvm *kvm, struct kvm_xen_gnttab *op) +{ + struct kvm_xen *xen = &kvm->arch.xen; + struct kvm_grant_table *gnttab = &xen->gnttab; + gfn_t *map = gnttab->frames_addr; + u64 gfn = op->grow.gfn; + u32 idx = op->grow.idx; + struct page *page; + + if (idx < gnttab->nr_frames || idx >= gnttab->max_nr_frames) + return -EINVAL; + + if (!idx && !gnttab->nr_frames && + !gnttab->initial) { + return -EINVAL; + } + + page = gfn_to_page(kvm, gfn); + if (is_error_page(page)) + return -EINVAL; + + map[idx] = gfn; + + gnttab->frames[idx] = page_to_virt(page); + if (!idx && !gnttab->nr_frames && + kvm_xen_gnttab_copy_initial_frame(kvm)) { + pr_err("kvm_xen: dom%u: failed to copy initial frame\n", + xen->domid); + return -EFAULT; + } + + if (kvm_xen_maptrack_grow(xen, gnttab->nr_frames + 1)) { + pr_warn("kvm_xen: dom%u: cannot grow maptrack\n", xen->domid); + return -EFAULT; + } + + pr_debug("kvm_xen: dom%u: grant table grow frames:%d/%d\n", xen->domid, + gnttab->nr_frames, gnttab->max_nr_frames); + return 0; +} + int kvm_vm_ioctl_xen_gnttab(struct kvm *kvm, struct kvm_xen_gnttab *op) { int r = -EINVAL; @@ -1174,6 +1261,9 @@ int kvm_vm_ioctl_xen_gnttab(struct kvm *kvm, struct kvm_xen_gnttab *op) case KVM_XEN_GNTTAB_F_INIT: r = kvm_xen_gnttab_init(kvm, &kvm->arch.xen, op, 0); break; + case KVM_XEN_GNTTAB_F_GROW: + r = kvm_xen_gnttab_grow(kvm, op); + break; default: r = -ENOSYS; break; diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index e4fb9bc34d61..ff7f7d019472 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1505,6 +1505,7 @@ struct kvm_xen_hvm_attr { } dom; struct kvm_xen_gnttab { #define KVM_XEN_GNTTAB_F_INIT 0 +#define KVM_XEN_GNTTAB_F_GROW (1 << 0) __u32 flags; union { struct { @@ -1512,6 +1513,10 @@ struct kvm_xen_hvm_attr { __u32 max_maptrack_frames; __u64 initial_frame; } init; + struct { + __u32 idx; + __u64 gfn; + } grow; __u32 padding[4]; }; } gnttab; From patchwork Wed Feb 20 20:15:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822681 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 32124922 for ; Wed, 20 Feb 2019 20:20:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 197A52F04E for ; Wed, 20 Feb 2019 20:20:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 178482F099; Wed, 20 Feb 2019 20:20:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2ED732F11E for ; Wed, 20 Feb 2019 20:20:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727443AbfBTUUs (ORCPT ); Wed, 20 Feb 2019 15:20:48 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:52916 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727869AbfBTUSa (ORCPT ); Wed, 20 Feb 2019 15:18:30 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8VMH081701; Wed, 20 Feb 2019 20:18:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=gQUPRbLoYWg0zdbNGHXgdQYJld3Oxus6PO4cvvWfLpw=; b=ELSXDtZNmmpI2AO7jtWQjgd+ax/0DCR4WA4LOFBi/appqSDAzNqe3A48BpjIoobJGdoq 9RlCnewbM46W+kT+NBFO2QpJy/AfIovF83BzDld/hZD1JS6b7BK9ZzI8jxkyFlXKB1Ih fwR9/Rju4gVmFw1zxBB+oC9Pr3RAQznuKOP6HoEXgdMX2edOXZtFUwtUWRI4gJKbolaF 1etmvRKW6uDliB8//ABYA0XJ6fo1PoymKZj0n7/NcZ9TZVEqQCrMGROndEtC5uyPY2Do 6n1sxAaVUjLM3aPuVWK97D8mSE2oFU4rXWXnY8W3/o7kSexDlR5FFlsLLOhCBpUlqPcI 1g== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2qp9xu3xm2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:16 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIAN8006676 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:10 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIABW009924; Wed, 20 Feb 2019 20:18:10 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:09 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 24/39] KVM: x86/xen: backend hypercall support Date: Wed, 20 Feb 2019 20:15:54 +0000 Message-Id: <20190220201609.28290-25-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ankur Arora Ordinarily a Xen backend domain would do hypercalls via int 0x81 (or vmcall) to enter a lower ring of execution. This is done via a hypercall_page which contains call stubs corresponding to each hypercall. For Xen backend driver support, however, we would like to do Xen hypercalls in the same ring. To that end we point the hypercall_page to a kvm owned text page which just does a local call (to kvm_xen_host_hcall().) Note, that this is different from hypercalls handled in kvm_xen_hypercall(), because the latter refers to domU hypercalls (so there is an actual drop in execution ring) while there isn't in kvm_xen_host_hcall(). Signed-off-by: Ankur Arora Signed-off-by: Joao Martins --- arch/x86/include/asm/kvm_host.h | 3 ++ arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/xen-asm.S | 66 +++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/xen.c | 68 +++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/xen.h | 4 +++ 5 files changed, 142 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kvm/xen-asm.S diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 70bb7339ddd4..55609e919e14 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1669,4 +1669,7 @@ static inline int kvm_cpu_get_apicid(int mps_cpu) #define put_smstate(type, buf, offset, val) \ *(type *)((buf) + (offset) - 0x7e00) = val +void kvm_xen_register_lcall(struct kvm_xen *shim); +void kvm_xen_unregister_lcall(void); + #endif /* _ASM_X86_KVM_HOST_H */ diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile index 2b46c93c9380..c1eaabbd0a54 100644 --- a/arch/x86/kvm/Makefile +++ b/arch/x86/kvm/Makefile @@ -10,7 +10,7 @@ kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o kvm-y += x86.o mmu.o emulate.o i8259.o irq.o lapic.o \ i8254.o ioapic.o irq_comm.o cpuid.o pmu.o mtrr.o \ - hyperv.o xen.o page_track.o debugfs.o + hyperv.o xen-asm.o xen.o page_track.o debugfs.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o vmx/evmcs.o vmx/nested.o kvm-amd-y += svm.o pmu_amd.o diff --git a/arch/x86/kvm/xen-asm.S b/arch/x86/kvm/xen-asm.S new file mode 100644 index 000000000000..10559fcfbe38 --- /dev/null +++ b/arch/x86/kvm/xen-asm.S @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2019 Oracle and/or its affiliates. All rights reserved. */ +#include +#include +#include +#include +#include +#include +#include +#include + + .balign PAGE_SIZE +ENTRY(kvm_xen_hypercall_page) + hcall=0 + .rept (PAGE_SIZE / 32) + FRAME_BEGIN + push %rcx /* Push call clobbered registers */ + push %r9 + push %r11 + mov $hcall, %rax + + call kvm_xen_host_hcall + pop %r11 + pop %r9 + pop %rcx + + FRAME_END + ret + .balign 32 + hcall = hcall + 1 + .endr +/* + * Hypercall symbols are used for unwinding the stack, so we give them names + * prefixed with kvm_xen_ (Xen hypercalls have symbols prefixed with xen_.) + */ +#define HYPERCALL(n) \ + .equ kvm_xen_hypercall_##n, kvm_xen_hypercall_page + __HYPERVISOR_##n * 32; \ + .type kvm_xen_hypercall_##n, @function; \ + .size kvm_xen_hypercall_##n, 32 +#include +#undef HYPERCALL +END(kvm_xen_hypercall_page) + +/* + * Some call stubs generated above do not have associated symbols. Generate + * bogus symbols for those hypercall blocks to stop objtool from complaining + * about unreachable code. + */ +.altmacro +.macro hypercall_missing N + .equ kvm_xen_hypercall_missing_\N, kvm_xen_hypercall_page + \N * 32; + .type kvm_xen_hypercall_missing_\N, @function; + .size kvm_xen_hypercall_missing_\N, 32; +.endm + +.macro hypercalls_missing N count=1 + .set n,\N + .rept \count + hypercall_missing %n + .set n,n+1 + .endr +.endm + +hypercalls_missing 11 1 +hypercalls_missing 42 6 +hypercalls_missing 56 72 diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 7266d27db210..645cd22ab4e7 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -19,6 +20,10 @@ #include #include #include +#include +#include +#include +#include #include "trace.h" @@ -43,6 +48,7 @@ struct evtchnfd { static int kvm_xen_evtchn_send(struct kvm_vcpu *vcpu, int port); static void *xen_vcpu_info(struct kvm_vcpu *v); static void kvm_xen_gnttab_free(struct kvm_xen *xen); +static int shim_hypercall(u64 code, u64 a0, u64 a1, u64 a2, u64 a3, u64 a4); #define XEN_DOMID_MIN 1 #define XEN_DOMID_MAX (DOMID_FIRST_RESERVED - 1) @@ -50,6 +56,9 @@ static void kvm_xen_gnttab_free(struct kvm_xen *xen); static rwlock_t domid_lock; static struct idr domid_to_kvm; +static struct hypercall_entry *hypercall_page_save; +static struct kvm_xen *xen_shim __read_mostly; + static int kvm_xen_domid_init(struct kvm *kvm, bool any, domid_t domid) { u16 min = XEN_DOMID_MIN, max = XEN_DOMID_MAX; @@ -1271,3 +1280,62 @@ int kvm_vm_ioctl_xen_gnttab(struct kvm *kvm, struct kvm_xen_gnttab *op) return r; } + +asmlinkage int kvm_xen_host_hcall(void) +{ + register unsigned long a0 asm(__HYPERCALL_RETREG); + register unsigned long a1 asm(__HYPERCALL_ARG1REG); + register unsigned long a2 asm(__HYPERCALL_ARG2REG); + register unsigned long a3 asm(__HYPERCALL_ARG3REG); + register unsigned long a4 asm(__HYPERCALL_ARG4REG); + register unsigned long a5 asm(__HYPERCALL_ARG5REG); + int ret; + + preempt_disable(); + ret = shim_hypercall(a0, a1, a2, a3, a4, a5); + preempt_enable(); + + return ret; +} + +void kvm_xen_register_lcall(struct kvm_xen *shim) +{ + hypercall_page_save = hypercall_page; + hypercall_page = kvm_xen_hypercall_page; + xen_shim = shim; +} +EXPORT_SYMBOL_GPL(kvm_xen_register_lcall); + +void kvm_xen_unregister_lcall(void) +{ + hypercall_page = hypercall_page_save; + hypercall_page_save = NULL; +} +EXPORT_SYMBOL_GPL(kvm_xen_unregister_lcall); + +static int shim_hcall_version(int op, struct xen_feature_info *fi) +{ + if (op != XENVER_get_features || !fi || fi->submap_idx != 0) + return -EINVAL; + + /* + * We need a limited set of features for a pseudo dom0. + */ + fi->submap = (1U << XENFEAT_auto_translated_physmap); + return 0; +} + +static int shim_hypercall(u64 code, u64 a0, u64 a1, u64 a2, u64 a3, u64 a4) +{ + int ret = -ENOSYS; + + switch (code) { + case __HYPERVISOR_xen_version: + ret = shim_hcall_version((int)a0, (void *)a1); + break; + default: + break; + } + + return ret; +} diff --git a/arch/x86/kvm/xen.h b/arch/x86/kvm/xen.h index 08ad4e1259df..9fa7c3dd111a 100644 --- a/arch/x86/kvm/xen.h +++ b/arch/x86/kvm/xen.h @@ -3,6 +3,8 @@ #ifndef __ARCH_X86_KVM_XEN_H__ #define __ARCH_X86_KVM_XEN_H__ +#include + static inline struct kvm_vcpu_xen *vcpu_to_xen_vcpu(struct kvm_vcpu *vcpu) { return &vcpu->arch.xen; @@ -48,4 +50,6 @@ int kvm_xen_has_pending_timer(struct kvm_vcpu *vcpu); void kvm_xen_inject_timer_irqs(struct kvm_vcpu *vcpu); bool kvm_xen_timer_enabled(struct kvm_vcpu *vcpu); +extern struct hypercall_entry kvm_xen_hypercall_page[128]; + #endif From patchwork Wed Feb 20 20:15:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822679 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DFA01922 for ; Wed, 20 Feb 2019 20:20:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C8F322F0EF for ; Wed, 20 Feb 2019 20:20:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BB50D2F0C6; Wed, 20 Feb 2019 20:20:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B45D62F112 for ; Wed, 20 Feb 2019 20:20:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727913AbfBTUSa (ORCPT ); Wed, 20 Feb 2019 15:18:30 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:52896 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727814AbfBTUS3 (ORCPT ); Wed, 20 Feb 2019 15:18:29 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8kOn081907; Wed, 20 Feb 2019 20:18:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=rivwbgHTX2wSH2CMj2+32JN9qlvvnpg8s+YqWIE/nys=; b=4+DQc80NisZW5DlVbkL3WRa8vN0EQ9B4I4Ipg+jlX2FBSgdLh9tnQAGc/nGMfsoVKGeT oTNzokvcNsmOxVfvXsQ4JZxAIlWY643iBZdOsYlFAr+0wImnE94JHyhMFd6HLRM3rVav auBL4uij7byatucZeNdqyLWCbeFaOPbT3Zn9YGRrAiLzq95G1EC7qZZxDT1xjfnkFwEU l1yGOMWN0x7F8bvuL/Lb6Jx5LOtCv3iCdFkBX49QFIKcMCraFMk8cTknhCbd84dcXsan OtcvYnlvzs3a4KouyJQ75tQjj3ECSWkPzY4f+MVaHrIiVa91hwKfBCdlRgfezomNr4tH YQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2130.oracle.com with ESMTP id 2qp9xu3xkx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:14 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIDx7019291 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:13 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKIDox018843; Wed, 20 Feb 2019 20:18:13 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:12 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 25/39] KVM: x86/xen: grant map support Date: Wed, 20 Feb 2019 20:15:55 +0000 Message-Id: <20190220201609.28290-26-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ankur Arora Introduce support for mapping grant references. The sequence of events to map a grant is: rframe = read_shared_entry(guest_grant_table, grant-ref); rpfn = get_user_pages_remote(remote_mm, rframe); mark_shared_entry(guest_grant_table, grant-ref, GTF_reading | GTF_writing); To correctly handle grant unmaps for mapped grants, we save the mapping parameters in maptrack. Also, grant map (and unmap) can be called from non-sleeping contexts, so we call get_user_pages_remote() in non-blocking mode and ask the user to retry. Also note that this code is not compliant with Xen's grant map/unmap ABI. In particular, we do not support multiple simultaneous mappings of a grant-reference. Later versions will support that. Co-developed-by: Joao Martins Signed-off-by: Ankur Arora Signed-off-by: Joao Martins --- arch/x86/kvm/xen.c | 396 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 396 insertions(+) diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 645cd22ab4e7..3603645086a7 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -9,6 +9,7 @@ #include "xen.h" #include "ioapic.h" +#include #include #include #include @@ -29,9 +30,11 @@ /* Grant v1 references per 4K page */ #define GPP_V1 (PAGE_SIZE / sizeof(struct grant_entry_v1)) +#define shared_entry(gt, ref) (&((gt)[(ref) / GPP_V1][(ref) % GPP_V1])) /* Grant mappings per 4K page */ #define MPP (PAGE_SIZE / sizeof(struct kvm_grant_map)) +#define maptrack_entry(mt, hdl) (&((mt)[(hdl) / MPP][(hdl) % MPP])) struct evtchnfd { struct eventfd_ctx *ctx; @@ -81,6 +84,18 @@ static int kvm_xen_domid_init(struct kvm *kvm, bool any, domid_t domid) return 0; } +static struct kvm *kvm_xen_find_vm(domid_t domid) +{ + unsigned long flags; + struct kvm *vm; + + read_lock_irqsave(&domid_lock, flags); + vm = idr_find(&domid_to_kvm, domid); + read_unlock_irqrestore(&domid_lock, flags); + + return vm; +} + int kvm_xen_free_domid(struct kvm *kvm) { struct kvm_xen *xen = &kvm->arch.xen; @@ -1153,7 +1168,20 @@ int kvm_xen_gnttab_init(struct kvm *kvm, struct kvm_xen *xen, gnttab->frames = addr; gnttab->frames[0] = xen->gnttab.initial; gnttab->max_nr_frames = max_frames; + + addr = kcalloc(max_mt_frames, sizeof(addr), GFP_KERNEL); + if (!addr) + goto out; + + /* Needs to be aligned at 16b boundary. */ + gnttab->handle = addr; gnttab->max_mt_frames = max_mt_frames; + + addr = (void *) get_zeroed_page(GFP_KERNEL); + if (!addr) + goto out; + gnttab->handle[0] = addr; + gnttab->nr_mt_frames = 1; gnttab->nr_frames = 0; @@ -1162,6 +1190,7 @@ int kvm_xen_gnttab_init(struct kvm *kvm, struct kvm_xen *xen, return 0; out: + kfree(xen->gnttab.handle); kfree(xen->gnttab.frames); kfree(xen->gnttab.frames_addr); if (page) @@ -1170,11 +1199,38 @@ int kvm_xen_gnttab_init(struct kvm *kvm, struct kvm_xen *xen, return -ENOMEM; } +static void kvm_xen_maptrack_free(struct kvm_xen *xen) +{ + u32 max_entries = xen->gnttab.nr_mt_frames * MPP; + struct kvm_grant_map *map; + int ref, inuse = 0; + + for (ref = 0; ref < max_entries; ref++) { + map = maptrack_entry(xen->gnttab.handle, ref); + + if (test_and_clear_bit(_KVM_GNTMAP_ACTIVE, + (unsigned long *)&map->flags)) { + put_page(virt_to_page(map->gpa)); + inuse++; + } + } + + if (inuse) + pr_debug("kvm: dom%u teardown %u mappings\n", + xen->domid, inuse); +} + void kvm_xen_gnttab_free(struct kvm_xen *xen) { struct kvm_grant_table *gnttab = &xen->gnttab; int i; + if (xen->domid) + kvm_xen_maptrack_free(xen); + + for (i = 0; i < gnttab->nr_mt_frames; i++) + free_page((unsigned long)gnttab->handle[i]); + for (i = 0; i < gnttab->nr_frames; i++) put_page(virt_to_page(gnttab->frames[i])); @@ -1313,6 +1369,343 @@ void kvm_xen_unregister_lcall(void) } EXPORT_SYMBOL_GPL(kvm_xen_unregister_lcall); +static inline int gnttab_entries(struct kvm *kvm) +{ + struct kvm_grant_table *gnttab = &kvm->arch.xen.gnttab; + int n = max_t(unsigned int, gnttab->nr_frames, 1); + + return n * ((n << PAGE_SHIFT) / sizeof(struct grant_entry_v1)); +} + +/* + * The first two members of a grant entry are updated as a combined pair. + * The following union allows that to happen in an endian-neutral fashion. + * Taken from Xen. + */ +union grant_combo { + uint32_t word; + struct { + uint16_t flags; + domid_t domid; + } shorts; +}; + +/* Marks a grant in use. Code largely borrowed from Xen. */ +static int set_grant_status(domid_t domid, bool readonly, + struct grant_entry_v1 *shah) +{ + int rc = GNTST_okay; + union grant_combo scombo, prev_scombo, new_scombo; + uint16_t mask = GTF_type_mask; + + /* + * We bound the number of times we retry CMPXCHG on memory locations + * that we share with a guest OS. The reason is that the guest can + * modify that location at a higher rate than we can + * read-modify-CMPXCHG, so the guest could cause us to livelock. There + * are a few cases where it is valid for the guest to race our updates + * (e.g., to change the GTF_readonly flag), so we allow a few retries + * before failing. + */ + int retries = 0; + + scombo.word = *(u32 *)shah; + + /* + * This loop attempts to set the access (reading/writing) flags + * in the grant table entry. It tries a cmpxchg on the field + * up to five times, and then fails under the assumption that + * the guest is misbehaving. + */ + for (;;) { + /* If not already pinned, check the grant domid and type. */ + if ((((scombo.shorts.flags & mask) != GTF_permit_access) || + (scombo.shorts.domid != domid))) { + rc = GNTST_general_error; + pr_err("Bad flags (%x) or dom (%d); expected d%d\n", + scombo.shorts.flags, scombo.shorts.domid, + domid); + return rc; + } + + new_scombo = scombo; + new_scombo.shorts.flags |= GTF_reading; + + if (!readonly) { + new_scombo.shorts.flags |= GTF_writing; + if (unlikely(scombo.shorts.flags & GTF_readonly)) { + rc = GNTST_general_error; + pr_err("Attempt to write-pin a r/o grant entry\n"); + return rc; + } + } + + prev_scombo.word = cmpxchg((u32 *)shah, + scombo.word, new_scombo.word); + if (likely(prev_scombo.word == scombo.word)) + break; + + if (retries++ == 4) { + rc = GNTST_general_error; + pr_err("Shared grant entry is unstable\n"); + return rc; + } + + scombo = prev_scombo; + } + + return rc; +} + +#define MT_HANDLE_DOMID_SHIFT 17 +#define MT_HANDLE_DOMID_MASK 0x7fff +#define MT_HANDLE_GREF_MASK 0x1ffff + +static u32 handle_get(domid_t domid, grant_ref_t ref) +{ + return (domid << MT_HANDLE_DOMID_SHIFT) | ref; +} + +static u16 handle_get_domid(grant_handle_t handle) +{ + return (handle >> MT_HANDLE_DOMID_SHIFT) & MT_HANDLE_DOMID_MASK; +} + +static grant_ref_t handle_get_grant(grant_handle_t handle) +{ + return handle & MT_HANDLE_GREF_MASK; +} + +static int map_grant_nosleep(struct kvm *rd, u64 frame, bool readonly, + struct page **page, u16 *err) +{ + unsigned long rhva; + int gup_flags, non_blocking; + int ret; + + *err = GNTST_general_error; + + if (!err || !page) + return -EINVAL; + + rhva = gfn_to_hva(rd, frame); + if (kvm_is_error_hva(rhva)) { + *err = GNTST_bad_page; + return -EFAULT; + } + + gup_flags = (readonly ? 0 : FOLL_WRITE) | FOLL_NOWAIT; + + /* get_user_pages will reset this were IO to be needed */ + non_blocking = 1; + + /* + * get_user_pages_*() family of functions can sleep if the page needs + * to be mapped in. However, our main consumer is the grant map + * hypercall and because we run in the same context as the caller + * (unlike a real hypercall) sleeping is not an option. + * + * This is how we avoid it: + * - sleeping on mmap_sem acquisition: we handle that by acquiring the + * read-lock before calling. + * If mmap_sem is contended, return with GNTST_eagain. + * - sync wait for pages to be swapped in: specify FOLL_NOWAIT. If IO + * was needed, would be returned via @non_blocking. Return + * GNTST_eagain if it is necessary and the user would retry. + * Also, in the blocking case, mmap_sem will be released + * asynchronously when the IO completes. + */ + ret = down_read_trylock(&rd->mm->mmap_sem); + if (ret == 0) { + *err = GNTST_eagain; + return -EBUSY; + } + + ret = get_user_pages_remote(rd->mm->owner, rd->mm, rhva, 1, gup_flags, + page, NULL, &non_blocking); + if (non_blocking) + up_read(&rd->mm->mmap_sem); + + if (ret == 1) { + *err = GNTST_okay; + } else if (ret == 0) { + *err = GNTST_eagain; + ret = -EBUSY; + } else if (ret < 0) { + pr_err("gnttab: failed to get pfn for hva %lx, err %d\n", + rhva, ret); + if (ret == -EFAULT) { + *err = GNTST_bad_page; + } else if (ret == -EBUSY) { + WARN_ON(non_blocking); + *err = GNTST_eagain; + } else { + *err = GNTST_general_error; + } + } + + return (ret >= 0) ? 0 : ret; +} + +static int shim_hcall_gntmap(struct kvm_xen *ld, + struct gnttab_map_grant_ref *op) +{ + struct kvm_grant_map map_old, map_new, *map = NULL; + bool readonly = op->flags & GNTMAP_readonly; + struct grant_entry_v1 *shah; + struct page *page = NULL; + unsigned long host_kaddr; + int err = -ENOSYS; + struct kvm *rd; + kvm_pfn_t rpfn; + u32 frame; + u32 idx; + + BUILD_BUG_ON(sizeof(*map) != 16); + + if (unlikely((op->host_addr))) { + pr_err("gnttab: bad host_addr %llx in map\n", op->host_addr); + op->status = GNTST_bad_virt_addr; + return 0; + } + + /* + * Make sure the guest does not try to smuggle any flags here + * (for instance _KVM_GNTMAP_ACTIVE.) + * The only allowable flag is GNTMAP_readonly. + */ + if (unlikely(op->flags & ~((u16) GNTMAP_readonly))) { + pr_err("gnttab: bad flags %x in map\n", op->flags); + op->status = GNTST_bad_gntref; + return 0; + } + + rd = kvm_xen_find_vm(op->dom); + if (unlikely(!rd)) { + pr_err("gnttab: could not find domain %u\n", op->dom); + op->status = GNTST_bad_domain; + return 0; + } + + if (unlikely(op->ref >= gnttab_entries(rd))) { + pr_err("gnttab: bad ref %u\n", op->ref); + op->status = GNTST_bad_gntref; + return 0; + } + + /* + * shah is potentially controlled by the user. We cache the frame but + * don't care about any changes to domid or flags since those get + * validated in set_grant_status() anyway. + * + * Note that if the guest changes the frame we will end up mapping the + * old frame. + */ + shah = shared_entry(rd->arch.xen.gnttab.frames_v1, op->ref); + frame = READ_ONCE(shah->frame); + + if (unlikely(shah->domid != ld->domid)) { + pr_err("gnttab: bad domain (%u != %u)\n", + shah->domid, ld->domid); + op->status = GNTST_bad_gntref; + goto out; + } + + idx = handle_get(op->dom, op->ref); + if (handle_get_grant(idx) < op->ref || + handle_get_domid(idx) < op->dom) { + pr_err("gnttab: out of maptrack entries (dom %u)\n", ld->domid); + op->status = GNTST_general_error; + goto out; + } + + map = maptrack_entry(rd->arch.xen.gnttab.handle, op->ref); + + /* + * Cache the old map value so we can do our checks on the stable + * version. Once the map is done, swap the mapping with the new map. + */ + map_old = *map; + if (map_old.flags & KVM_GNTMAP_ACTIVE) { + pr_err("gnttab: grant ref %u dom %u in use\n", + op->ref, ld->domid); + op->status = GNTST_bad_gntref; + goto out; + } + + err = map_grant_nosleep(rd, frame, readonly, &page, &op->status); + if (err) { + if (err != -EBUSY) + op->status = GNTST_bad_gntref; + goto out; + } + + err = set_grant_status(ld->domid, readonly, shah); + if (err != GNTST_okay) { + pr_err("gnttab: pin failed\n"); + put_page(page); + op->status = err; + goto out; + } + + rpfn = page_to_pfn(page); + host_kaddr = (unsigned long) pfn_to_kaddr(rpfn); + + map_new.domid = op->dom; + map_new.ref = op->ref; + map_new.flags = op->flags; + map_new.gpa = host_kaddr; + + map_new.flags |= KVM_GNTMAP_ACTIVE; + + /* + * Protect against a grant-map that could come in between our check for + * KVM_GNTMAP_ACTIVE above and assuming the ownership of the mapping. + * + * Use cmpxchg_double() so we can update mapping atomically (which + * luckily fits in 16b.) + */ + if (cmpxchg_double(&map->gpa, &map->fields, + map_old.gpa, map_old.fields, + map_new.gpa, map_new.fields) == false) { + put_page(page); + op->status = GNTST_bad_gntref; + goto out; + } + + op->dev_bus_addr = rpfn << PAGE_SHIFT; + op->handle = idx; + op->status = GNTST_okay; + op->host_addr = host_kaddr; + return 0; + +out: + /* The error code is stored in @status. */ + return 0; +} + +static int shim_hcall_gnttab(int op, void *p, int count) +{ + int ret = -ENOSYS; + int i; + + switch (op) { + case GNTTABOP_map_grant_ref: { + struct gnttab_map_grant_ref *ref = p; + + for (i = 0; i < count; i++) + shim_hcall_gntmap(xen_shim, ref + i); + ret = 0; + break; + } + default: + pr_info("lcall-gnttab:op default=%d\n", op); + break; + } + + return ret; +} + static int shim_hcall_version(int op, struct xen_feature_info *fi) { if (op != XENVER_get_features || !fi || fi->submap_idx != 0) @@ -1330,6 +1723,9 @@ static int shim_hypercall(u64 code, u64 a0, u64 a1, u64 a2, u64 a3, u64 a4) int ret = -ENOSYS; switch (code) { + case __HYPERVISOR_grant_table_op: + ret = shim_hcall_gnttab((int) a0, (void *) a1, (int) a2); + break; case __HYPERVISOR_xen_version: ret = shim_hcall_version((int)a0, (void *)a1); break; From patchwork Wed Feb 20 20:15:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822675 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 39DBF1399 for ; Wed, 20 Feb 2019 20:20:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 25B732F06C for ; Wed, 20 Feb 2019 20:20:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 19B2A29D82; Wed, 20 Feb 2019 20:20:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A1B532EABB for ; Wed, 20 Feb 2019 20:20:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727670AbfBTUUU (ORCPT ); Wed, 20 Feb 2019 15:20:20 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:36216 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727972AbfBTUSg (ORCPT ); Wed, 20 Feb 2019 15:18:36 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8YIC087297; Wed, 20 Feb 2019 20:18:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=YZ80s2dOqvHZLSQICuv37RTvDVLvQVdgaR+I/O+Ppao=; b=TawXR2tZtIHJQg0fJ3iWk9NXhfNqK5+2LgqSUautIUcoNAtdrcFluwEI6ZfWFP5EbiRY wL7/QhsLf7ie6hBe3eSbpl5bcFsLS71iLDivSHoL1rxNQYNSn6UGBJJEpIe5SvjGEYqa ZBNcyzvGETuQA9qPZ4RlU4Fy8Yi5Qj5IASIA3EaTo/AJWntzh+dekpEZLtu0LDFvrKhI hwIt+GSrLprs8zU7P7s1yGPPioYs/a5bcRCAuLgxJHyDQb3IlrlgvJdXlhPi5LuqrlM1 tuGuLCibtTpQ5jdGg+Z9gyogAFQht172H1iYiZUeFFvnHnt7futsI7F+uJ+rP9as+8Kt 1g== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2qpb5rktuj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:23 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIHmo024795 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:17 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKIGjM015848; Wed, 20 Feb 2019 20:18:16 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:16 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 26/39] KVM: x86/xen: grant unmap support Date: Wed, 20 Feb 2019 20:15:56 +0000 Message-Id: <20190220201609.28290-27-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=803 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ankur Arora Grant unmap removes the grant from maptrack and marks it as not in use. We maintain a one-to-one correspondence between grant table and maptrack entries so there's no contention in allocation/free. Co-developed-by: Joao Martins Signed-off-by: Ankur Arora Signed-off-by: Joao Martins --- arch/x86/kvm/xen.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 3603645086a7..8f06924e0dfa 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -1684,6 +1684,69 @@ static int shim_hcall_gntmap(struct kvm_xen *ld, return 0; } +static int shim_hcall_gntunmap(struct kvm_xen *xen, + struct gnttab_unmap_grant_ref *op) +{ + struct kvm_grant_map *map, unmap; + struct grant_entry_v1 **rgt; + struct grant_entry_v1 *shah; + struct kvm *rd = NULL; + domid_t domid; + u32 ref; + + domid = handle_get_domid(op->handle); + ref = handle_get_grant(op->handle); + + + rd = kvm_xen_find_vm(domid); + if (unlikely(!rd)) { + /* We already teardown all ongoing grant maps */ + op->status = GNTST_okay; + return 0; + } + + if (unlikely(ref >= gnttab_entries(rd))) { + pr_err("gnttab: bad ref %u\n", ref); + op->status = GNTST_bad_handle; + return 0; + } + + rgt = rd->arch.xen.gnttab.frames_v1; + map = maptrack_entry(rd->arch.xen.gnttab.handle, ref); + + /* + * The test_and_clear_bit (below) serializes ownership of this + * grant-entry. After we clear it, there can be a grant-map on this + * entry. So we cache the unmap entry before relinquishing ownership. + */ + unmap = *map; + + if (!test_and_clear_bit(_KVM_GNTMAP_ACTIVE, + (unsigned long *) &map->flags)) { + pr_err("gnttab: bad flags for %u (dom %u ref %u) flags %x\n", + op->handle, domid, ref, unmap.flags); + op->status = GNTST_bad_handle; + return 0; + } + + /* Give up the reference taken in get_user_pages_remote(). */ + put_page(virt_to_page(unmap.gpa)); + + shah = shared_entry(rgt, unmap.ref); + + /* + * We have cleared _KVM_GNTMAP_ACTIVE, so a simultaneous grant-map + * could update the shah and we would stomp all over it but the + * guest deserves it. + */ + if (!(unmap.flags & GNTMAP_readonly)) + clear_bit(_GTF_writing, (unsigned long *) &shah->flags); + clear_bit(_GTF_reading, (unsigned long *) &shah->flags); + + op->status = GNTST_okay; + return 0; +} + static int shim_hcall_gnttab(int op, void *p, int count) { int ret = -ENOSYS; @@ -1698,6 +1761,16 @@ static int shim_hcall_gnttab(int op, void *p, int count) ret = 0; break; } + case GNTTABOP_unmap_grant_ref: { + struct gnttab_unmap_grant_ref *ref = p; + + for (i = 0; i < count; i++) { + shim_hcall_gntunmap(xen_shim, ref + i); + ref[i].host_addr = 0; + } + ret = 0; + break; + } default: pr_info("lcall-gnttab:op default=%d\n", op); break; From patchwork Wed Feb 20 20:15:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822645 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4212F1399 for ; Wed, 20 Feb 2019 20:18:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2EAA62E97D for ; Wed, 20 Feb 2019 20:18:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1F9562EABE; Wed, 20 Feb 2019 20:18:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8FE612E97D for ; Wed, 20 Feb 2019 20:18:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727980AbfBTUSh (ORCPT ); Wed, 20 Feb 2019 15:18:37 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:33848 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727968AbfBTUSe (ORCPT ); Wed, 20 Feb 2019 15:18:34 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8VmG087414; Wed, 20 Feb 2019 20:18:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=87hDNam+EDhaaTcyC+shLftB4Uu5fmCyLj6CCzNbwoI=; b=jCdyIhHnnd59kr48KSPhb/CVx/UxspTKtJV5zW9PyFt1J9cwuTeCl9ejpJe201zxrvK5 vMMg+hAr8H1NvhxDRbQ9iMPrcqrcl6BYwTtU6mhLCiTIoMQMzMysyLfP5tLz1DOx7pRT onwKSebJQSHBupbj4BK/RG7uWFnZK2ZS0xDOi3U1M8LMzhqZzW2sbfudQ4znI9uEUk26 B8wU5PB80ldae+LhlQTnbUSJ7a9NYxllkz7GEhNTQGJVLvFd07RfuLP3f9U/0iqyex3j uGLiPhGWMFk0vXafNzarMrzH2p0KAph/dlhi4YUPR7Wtyh7RYV3Ehv2XOcA/HjSPLLIg 9w== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2qp81ec3df-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:20 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIJ1v024885 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:19 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIJTA009962; Wed, 20 Feb 2019 20:18:19 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:19 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 27/39] KVM: x86/xen: grant copy support Date: Wed, 20 Feb 2019 20:15:57 +0000 Message-Id: <20190220201609.28290-28-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Copies from source grant reference to dest grant reference (or the other way around.) Signed-off-by: Joao Martins --- arch/x86/kvm/xen.c | 151 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 151 insertions(+) diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 8f06924e0dfa..fecc548b2f12 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -10,6 +10,7 @@ #include "ioapic.h" #include +#include #include #include #include @@ -1747,6 +1748,148 @@ static int shim_hcall_gntunmap(struct kvm_xen *xen, return 0; } +static unsigned long __kvm_gfn_to_hva(struct kvm_vcpu *vcpu, gfn_t gfn) +{ + struct kvm_xen *xen = vcpu ? &vcpu->kvm->arch.xen : xen_shim; + unsigned long hva; + + if (xen->domid == 0) + return (unsigned long) page_to_virt(pfn_to_page(gfn)); + + hva = gfn_to_hva(vcpu->kvm, gfn); + if (unlikely(kvm_is_error_hva(hva))) + return 0; + + return hva; +} + +static int __kvm_gref_to_page(struct kvm_vcpu *vcpu, grant_ref_t ref, + domid_t domid, struct page **page, + int16_t *status) +{ + struct kvm_xen *source = vcpu ? &vcpu->kvm->arch.xen : xen_shim; + struct grant_entry_v1 *shah; + struct grant_entry_v1 **gt; + struct kvm *dest; + + dest = kvm_xen_find_vm(domid); + if (unlikely(!dest)) { + pr_err("gnttab: could not find domain %u\n", domid); + *status = GNTST_bad_domain; + return 0; + } + + if (unlikely(ref >= gnttab_entries(dest))) { + pr_err("gnttab: bad ref %u\n", ref); + *status = GNTST_bad_gntref; + return 0; + } + + gt = dest->arch.xen.gnttab.frames_v1; + shah = shared_entry(gt, ref); + if (unlikely(shah->domid != source->domid)) { + pr_err("gnttab: bad domain (%u != %u)\n", + shah->domid, source->domid); + *status = GNTST_bad_gntref; + return 0; + } + + (void) map_grant_nosleep(dest, shah->frame, 0, page, status); + + return 0; +} + +static int shim_hcall_gntcopy(struct kvm_vcpu *vcpu, + struct gnttab_copy *op) +{ + void *saddr = NULL, *daddr = NULL; + struct page *spage = NULL, *dpage = NULL; + unsigned long hva; + int err = -ENOSYS; + gfn_t gfn; + + if (!(op->flags & GNTCOPY_source_gref) && + (op->source.domid == DOMID_SELF)) { + gfn = op->source.u.gmfn; + hva = __kvm_gfn_to_hva(vcpu, gfn); + if (unlikely(!hva)) { + pr_err("gnttab: bad source gfn:%llx\n", gfn); + op->status = GNTST_general_error; + err = 0; + return 0; + } + + saddr = (void *) (((unsigned long) hva) + op->source.offset); + } else if (op->flags & GNTCOPY_source_gref) { + op->status = GNTST_okay; + if (__kvm_gref_to_page(vcpu, op->source.u.ref, + op->source.domid, &spage, &op->status)) + return -EFAULT; + + if (!spage || op->status != GNTST_okay) { + pr_err("gnttab: failed to get page for source gref:%x\n", + op->source.u.ref); + err = 0; + goto out; + } + + saddr = kmap(spage); + saddr = (void *) (((unsigned long) saddr) + op->source.offset); + } + + if (!(op->flags & GNTCOPY_dest_gref) && + (op->dest.domid == DOMID_SELF)) { + gfn = op->dest.u.gmfn; + hva = __kvm_gfn_to_hva(vcpu, gfn); + if (unlikely(!hva)) { + pr_err("gnttab: bad dest gfn:%llx\n", gfn); + op->status = GNTST_general_error; + err = 0; + return 0; + } + + daddr = (void *) (((unsigned long) hva) + op->dest.offset); + } else if (op->flags & GNTCOPY_dest_gref) { + op->status = GNTST_okay; + if (__kvm_gref_to_page(vcpu, op->dest.u.ref, + op->dest.domid, &dpage, &op->status)) + return -EFAULT; + + if (!dpage || op->status != GNTST_okay) { + pr_err("gnttab: failed to get page for dest gref:%x\n", + op->dest.u.ref); + err = 0; + goto out; + } + + daddr = kmap(dpage); + daddr = (void *) (((unsigned long) daddr) + op->dest.offset); + } + + if (unlikely(!daddr || !saddr)) { + op->status = GNTST_general_error; + err = 0; + goto out; + } + + memcpy(daddr, saddr, op->len); + + if (spage) + kunmap(spage); + if (dpage) + kunmap(dpage); + + + err = 0; + op->status = GNTST_okay; +out: + if (spage) + put_page(spage); + if (dpage) + put_page(dpage); + return err; +} + static int shim_hcall_gnttab(int op, void *p, int count) { int ret = -ENOSYS; @@ -1771,6 +1914,14 @@ static int shim_hcall_gnttab(int op, void *p, int count) ret = 0; break; } + case GNTTABOP_copy: { + struct gnttab_copy *op = p; + + for (i = 0; i < count; i++) + shim_hcall_gntcopy(NULL, op + i); + ret = 0; + break; + } default: pr_info("lcall-gnttab:op default=%d\n", op); break; From patchwork Wed Feb 20 20:15:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822647 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DA4E71399 for ; Wed, 20 Feb 2019 20:18:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C25772E97D for ; Wed, 20 Feb 2019 20:18:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B38F32EABE; Wed, 20 Feb 2019 20:18:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B61A72E97D for ; Wed, 20 Feb 2019 20:18:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728013AbfBTUSl (ORCPT ); Wed, 20 Feb 2019 15:18:41 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:36236 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727932AbfBTUSi (ORCPT ); Wed, 20 Feb 2019 15:18:38 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8VOd087139; Wed, 20 Feb 2019 20:18:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=34dhGVmA7cwyWHVj6W3v3cz50sdqag95jPIz8PaaXqQ=; b=micbrxhSej3rnBFycSvbC8DEka9LUAG+T72t3VwzIWiXeFar1GSjuG5uOaWhhhEWTvRm gN+0NTsyt8o8LqxjD+ErrrksjHsYO1nUZQPWku+CwYv7/rIH99z+lpLYNaSNaYWP98nK sOB+Gw9RrYfqc/bv8g2OlE9mfrM9kgGQpq5A3IF6jHNiG2SZ8U8sPvqqS1fvbpoZjY5w 9YIYZGE6mzmZnVDEVkPBfvky81NPWRt/nIOFW1sXb8Jqdw2SSEEfyMYwoSna3Bdym7OY FsVQ7QuMKk/Ksd37jfKIHDXUb+ACe+OCjfTq+3W4Jc9VlnwgY37smcRY/6C2KJE6YbFK qA== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2qpb5rktum-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:23 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIMvj019690 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:23 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIM81009979; Wed, 20 Feb 2019 20:18:22 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:22 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 28/39] KVM: x86/xen: interdomain evtchn support Date: Wed, 20 Feb 2019 20:15:58 +0000 Message-Id: <20190220201609.28290-29-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ankur Arora Implement sending events between backend and the guest. To send an event we mark the event channel pending by setting some bits in the shared_info and vcpu_info pages and deliver the upcall on the destination vcpu. To send an event to dom0, we mark the event channel pending and send an IPI to the destination vcpu, which would invoke the event channel upcall handler, which inturn calls the ISR registered by the backend drivers. When sending to the guest we fetch the vcpu from the guest, mark the event channel pending and deliver the interrupt to the guest. Co-developed-by: Joao Martins Signed-off-by: Ankur Arora Signed-off-by: Joao Martins --- arch/x86/kvm/xen.c | 271 ++++++++++++++++++++++++++++++++++++++++++++--- include/uapi/linux/kvm.h | 10 +- 2 files changed, 263 insertions(+), 18 deletions(-) diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index fecc548b2f12..420e3ebb66bc 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -27,6 +27,10 @@ #include #include +#include +#include +#include + #include "trace.h" /* Grant v1 references per 4K page */ @@ -46,12 +50,18 @@ struct evtchnfd { struct { u8 type; } virq; + struct { + domid_t dom; + struct kvm *vm; + u32 port; + } remote; }; }; static int kvm_xen_evtchn_send(struct kvm_vcpu *vcpu, int port); -static void *xen_vcpu_info(struct kvm_vcpu *v); +static void *vcpu_to_xen_vcpu_info(struct kvm_vcpu *v); static void kvm_xen_gnttab_free(struct kvm_xen *xen); +static int kvm_xen_evtchn_send_shim(struct kvm_xen *shim, struct evtchnfd *evt); static int shim_hypercall(u64 code, u64 a0, u64 a1, u64 a2, u64 a3, u64 a4); #define XEN_DOMID_MIN 1 @@ -114,7 +124,7 @@ int kvm_xen_free_domid(struct kvm *kvm) int kvm_xen_has_interrupt(struct kvm_vcpu *vcpu) { struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(vcpu); - struct vcpu_info *vcpu_info = xen_vcpu_info(vcpu); + struct vcpu_info *vcpu_info = vcpu_to_xen_vcpu_info(vcpu); if (!!atomic_read(&vcpu_xen->cb.queued) || (vcpu_info && test_bit(0, (unsigned long *) &vcpu_info->evtchn_upcall_pending))) @@ -386,7 +396,7 @@ static int kvm_xen_shared_info_init(struct kvm *kvm, gfn_t gfn) return 0; } -static void *xen_vcpu_info(struct kvm_vcpu *v) +static void *vcpu_to_xen_vcpu_info(struct kvm_vcpu *v) { struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(v); struct kvm_xen *kvm = &v->kvm->arch.xen; @@ -478,7 +488,7 @@ void kvm_xen_setup_pvclock_page(struct kvm_vcpu *v) { struct kvm_vcpu_xen *vcpu_xen = vcpu_to_xen_vcpu(v); struct pvclock_vcpu_time_info *guest_hv_clock; - void *hva = xen_vcpu_info(v); + void *hva = vcpu_to_xen_vcpu_info(v); unsigned int offset; offset = offsetof(struct vcpu_info, time); @@ -638,8 +648,6 @@ static int kvm_xen_evtchn_2l_set_pending(struct shared_info *shared_info, return 1; } -#undef BITS_PER_EVTCHN_WORD - static int kvm_xen_evtchn_set_pending(struct kvm_vcpu *svcpu, struct evtchnfd *evfd) { @@ -670,8 +678,44 @@ static void kvm_xen_check_poller(struct kvm_vcpu *vcpu, int port) wake_up(&vcpu_xen->sched_waitq); } +static void kvm_xen_evtchn_2l_reset_port(struct shared_info *shared_info, + int port) +{ + clear_bit(port, (unsigned long *) shared_info->evtchn_pending); + clear_bit(port, (unsigned long *) shared_info->evtchn_mask); +} + +static inline struct evtchnfd *port_to_evtchn(struct kvm *kvm, int port) +{ + struct kvm_xen *xen = kvm ? &kvm->arch.xen : xen_shim; + + return idr_find(&xen->port_to_evt, port); +} + +static struct kvm_vcpu *get_remote_vcpu(struct evtchnfd *source) +{ + struct kvm *rkvm = source->remote.vm; + int rport = source->remote.port; + struct evtchnfd *dest = NULL; + struct kvm_vcpu *vcpu = NULL; + + WARN_ON(source->type <= XEN_EVTCHN_TYPE_IPI); + + if (!rkvm) + return NULL; + + /* conn_to_evt is protected by vcpu->kvm->srcu */ + dest = port_to_evtchn(rkvm, rport); + if (!dest) + return NULL; + + vcpu = kvm_get_vcpu(rkvm, dest->vcpu); + return vcpu; +} + static int kvm_xen_evtchn_send(struct kvm_vcpu *vcpu, int port) { + struct kvm_vcpu *target = vcpu; struct eventfd_ctx *eventfd; struct evtchnfd *evtchnfd; @@ -680,10 +724,19 @@ static int kvm_xen_evtchn_send(struct kvm_vcpu *vcpu, int port) if (!evtchnfd) return -ENOENT; + if (evtchnfd->type == XEN_EVTCHN_TYPE_INTERDOM || + evtchnfd->type == XEN_EVTCHN_TYPE_UNBOUND) { + target = get_remote_vcpu(evtchnfd); + port = evtchnfd->remote.port; + + if (!target && !evtchnfd->remote.dom) + return kvm_xen_evtchn_send_shim(xen_shim, evtchnfd); + } + eventfd = evtchnfd->ctx; - if (!kvm_xen_evtchn_set_pending(vcpu, evtchnfd)) { + if (!kvm_xen_evtchn_set_pending(target, evtchnfd)) { if (!eventfd) - kvm_xen_evtchnfd_upcall(vcpu, evtchnfd); + kvm_xen_evtchnfd_upcall(target, evtchnfd); else eventfd_signal(eventfd, 1); } @@ -894,6 +947,67 @@ static int kvm_xen_hcall_sched_op(struct kvm_vcpu *vcpu, int cmd, u64 param) return ret; } +static void kvm_xen_call_function_deliver(void *_) +{ + xen_hvm_evtchn_do_upcall(); +} + +static inline int kvm_xen_evtchn_call_function(struct evtchnfd *event) +{ + int ret; + + if (!irqs_disabled()) + return smp_call_function_single(event->vcpu, + kvm_xen_call_function_deliver, + NULL, 0); + + local_irq_enable(); + ret = smp_call_function_single(event->vcpu, + kvm_xen_call_function_deliver, NULL, 0); + local_irq_disable(); + + return ret; +} + +static int kvm_xen_evtchn_send_shim(struct kvm_xen *dom0, struct evtchnfd *e) +{ + struct shared_info *s = HYPERVISOR_shared_info; + struct evtchnfd *remote; + int pending; + + remote = idr_find(&dom0->port_to_evt, e->remote.port); + if (!remote) + return -ENOENT; + + pending = kvm_xen_evtchn_2l_set_pending(s, + per_cpu(xen_vcpu, remote->vcpu), + remote->port); + return kvm_xen_evtchn_call_function(remote); +} + +static int __kvm_xen_evtchn_send_guest(struct kvm_vcpu *vcpu, int port) +{ + struct evtchnfd *evtchnfd; + struct eventfd_ctx *eventfd; + + /* conn_to_evt is protected by vcpu->kvm->srcu */ + evtchnfd = idr_find(&vcpu->kvm->arch.xen.port_to_evt, port); + if (!evtchnfd) + return -ENOENT; + + eventfd = evtchnfd->ctx; + if (!kvm_xen_evtchn_set_pending(vcpu, evtchnfd)) + kvm_xen_evtchnfd_upcall(vcpu, evtchnfd); + + kvm_xen_check_poller(kvm_get_vcpu(vcpu->kvm, evtchnfd->vcpu), port); + return 0; +} + +static int kvm_xen_evtchn_send_guest(struct evtchnfd *evt, int port) +{ + return __kvm_xen_evtchn_send_guest(get_remote_vcpu(evt), port); +} + int kvm_xen_hypercall(struct kvm_vcpu *vcpu) { bool longmode; @@ -1045,13 +1159,15 @@ static int kvm_xen_eventfd_update(struct kvm *kvm, struct idr *port_to_evt, return 0; } -static int kvm_xen_eventfd_assign(struct kvm *kvm, struct idr *port_to_evt, - struct mutex *port_lock, - struct kvm_xen_eventfd *args) +int kvm_xen_eventfd_assign(struct kvm *kvm, struct idr *port_to_evt, + struct mutex *port_lock, + struct kvm_xen_eventfd *args) { + struct evtchnfd *evtchnfd, *unbound = NULL; struct eventfd_ctx *eventfd = NULL; - struct evtchnfd *evtchnfd; + struct kvm *remote_vm = NULL; u32 port = args->port; + u32 endport = 0; int ret; if (args->fd != -1) { @@ -1064,25 +1180,56 @@ static int kvm_xen_eventfd_assign(struct kvm *kvm, struct idr *port_to_evt, args->virq.type >= KVM_XEN_NR_VIRQS) return -EINVAL; + if (args->remote.domid == DOMID_SELF) + remote_vm = kvm; + else if (args->remote.domid == xen_shim->domid) + remote_vm = NULL; + else if ((args->type == XEN_EVTCHN_TYPE_INTERDOM || + args->type == XEN_EVTCHN_TYPE_UNBOUND)) { + remote_vm = kvm_xen_find_vm(args->remote.domid); + if (!remote_vm) + return -ENOENT; + } + + if (args->type == XEN_EVTCHN_TYPE_INTERDOM) { + unbound = port_to_evtchn(remote_vm, args->remote.port); + if (!unbound) + return -ENOENT; + } + evtchnfd = kzalloc(sizeof(struct evtchnfd), GFP_KERNEL); if (!evtchnfd) return -ENOMEM; evtchnfd->ctx = eventfd; - evtchnfd->port = port; evtchnfd->vcpu = args->vcpu; evtchnfd->type = args->type; + if (evtchnfd->type == XEN_EVTCHN_TYPE_VIRQ) evtchnfd->virq.type = args->virq.type; + else if ((evtchnfd->type == XEN_EVTCHN_TYPE_UNBOUND) || + (evtchnfd->type == XEN_EVTCHN_TYPE_INTERDOM)) { + evtchnfd->remote.dom = args->remote.domid; + evtchnfd->remote.vm = remote_vm; + evtchnfd->remote.port = args->remote.port; + } + + if (port == 0) + port = 1; /* evtchns in range (0..INT_MAX] */ + else + endport = port + 1; mutex_lock(port_lock); - ret = idr_alloc(port_to_evt, evtchnfd, port, port + 1, + ret = idr_alloc(port_to_evt, evtchnfd, port, endport, GFP_KERNEL); mutex_unlock(port_lock); if (ret >= 0) { - if (evtchnfd->type == XEN_EVTCHN_TYPE_VIRQ) + evtchnfd->port = args->port = ret; + if (kvm && evtchnfd->type == XEN_EVTCHN_TYPE_VIRQ) kvm_xen_set_virq(kvm, evtchnfd); + else if (evtchnfd->type == XEN_EVTCHN_TYPE_INTERDOM) + unbound->remote.port = ret; return 0; } @@ -1107,8 +1254,14 @@ static int kvm_xen_eventfd_deassign(struct kvm *kvm, struct idr *port_to_evt, if (!evtchnfd) return -ENOENT; - if (kvm) + if (!kvm) { + struct shared_info *shinfo = HYPERVISOR_shared_info; + + kvm_xen_evtchn_2l_reset_port(shinfo, port); + } else { synchronize_srcu(&kvm->srcu); + } + if (evtchnfd->ctx) eventfd_ctx_put(evtchnfd->ctx); kfree(evtchnfd); @@ -1930,6 +2083,89 @@ static int shim_hcall_gnttab(int op, void *p, int count) return ret; } +static int shim_hcall_evtchn_send(struct kvm_xen *dom0, struct evtchn_send *snd) +{ + struct evtchnfd *event; + + event = idr_find(&dom0->port_to_evt, snd->port); + if (!event) + return -ENOENT; + + if (event->remote.vm == NULL) + return kvm_xen_evtchn_send_shim(xen_shim, event); + else if (event->type == XEN_EVTCHN_TYPE_INTERDOM || + event->type == XEN_EVTCHN_TYPE_UNBOUND) + return kvm_xen_evtchn_send_guest(event, event->remote.port); + else + return -EINVAL; + + return 0; +} + +static int shim_hcall_evtchn(int op, void *p) +{ + int ret; + struct kvm_xen_eventfd evt; + + if (p == NULL) + return -EINVAL; + + memset(&evt, 0, sizeof(evt)); + + switch (op) { + case EVTCHNOP_bind_interdomain: { + struct evtchn_bind_interdomain *un; + + un = (struct evtchn_bind_interdomain *) p; + + evt.fd = -1; + evt.port = 0; + if (un->remote_port == 0) { + evt.type = XEN_EVTCHN_TYPE_UNBOUND; + evt.remote.domid = un->remote_dom; + } else { + evt.type = XEN_EVTCHN_TYPE_INTERDOM; + evt.remote.domid = un->remote_dom; + evt.remote.port = un->remote_port; + } + + ret = kvm_xen_eventfd_assign(NULL, &xen_shim->port_to_evt, + &xen_shim->xen_lock, &evt); + un->local_port = evt.port; + break; + } + case EVTCHNOP_alloc_unbound: { + struct evtchn_alloc_unbound *un; + + un = (struct evtchn_alloc_unbound *) p; + + if (un->dom != DOMID_SELF || un->remote_dom != DOMID_SELF) + return -EINVAL; + evt.fd = -1; + evt.port = 0; + evt.type = XEN_EVTCHN_TYPE_UNBOUND; + evt.remote.domid = DOMID_SELF; + + ret = kvm_xen_eventfd_assign(NULL, &xen_shim->port_to_evt, + &xen_shim->xen_lock, &evt); + un->port = evt.port; + break; + } + case EVTCHNOP_send: { + struct evtchn_send *send; + + send = (struct evtchn_send *) p; + ret = shim_hcall_evtchn_send(xen_shim, send); + break; + } + default: + ret = -EINVAL; + break; + } + + return ret; +} + static int shim_hcall_version(int op, struct xen_feature_info *fi) { if (op != XENVER_get_features || !fi || fi->submap_idx != 0) @@ -1947,6 +2183,9 @@ static int shim_hypercall(u64 code, u64 a0, u64 a1, u64 a2, u64 a3, u64 a4) int ret = -ENOSYS; switch (code) { + case __HYPERVISOR_event_channel_op: + ret = shim_hcall_evtchn((int) a0, (void *)a1); + break; case __HYPERVISOR_grant_table_op: ret = shim_hcall_gnttab((int) a0, (void *) a1, (int) a2); break; diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index ff7f7d019472..74d877792dfa 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1483,8 +1483,10 @@ struct kvm_xen_hvm_attr { } vcpu_attr; struct kvm_xen_eventfd { -#define XEN_EVTCHN_TYPE_VIRQ 0 -#define XEN_EVTCHN_TYPE_IPI 1 +#define XEN_EVTCHN_TYPE_VIRQ 0 +#define XEN_EVTCHN_TYPE_IPI 1 +#define XEN_EVTCHN_TYPE_INTERDOM 2 +#define XEN_EVTCHN_TYPE_UNBOUND 3 __u32 type; __u32 port; __u32 vcpu; @@ -1497,6 +1499,10 @@ struct kvm_xen_hvm_attr { struct { __u8 type; } virq; + struct { + __u16 domid; + __u32 port; + } remote; __u32 padding[2]; }; } evtchn; From patchwork Wed Feb 20 20:15:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822669 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 857ED922 for ; Wed, 20 Feb 2019 20:20:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6BCA52F14F for ; Wed, 20 Feb 2019 20:20:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A89F72F0EF; Wed, 20 Feb 2019 20:20:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2EFF42F17A for ; Wed, 20 Feb 2019 20:20:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728002AbfBTUSs (ORCPT ); Wed, 20 Feb 2019 15:18:48 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:53128 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728003AbfBTUSl (ORCPT ); Wed, 20 Feb 2019 15:18:41 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8Uhf081679; Wed, 20 Feb 2019 20:18:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=ObvMkIO580oz+cYvFFnQV9+Yfz6Iys4YkrQlTPQzc0g=; b=ijkbP7HLCyY94yv4cDu+h4kxzcpki4HqC431yvgCZOoK72QjWVl6RIYmthE0Jfmepdbk qFuRHDXxleVoa9Fq7/eJdpm9LT/rq5H4v16dBci38UNVfvGsZyVBuEKIqMclIvXus7nQ 6QJ/x+MTR6oLAu6SL5hKa1GoUQPWR70FjGdTz3l/exZ6u1TQ14XFhU4Yg4yNuy3Zl3VS OWEBCbNQToKGrW3hzJw4CaDJYyR9JdWSYUYCHYZQHzS0sqkzNyfuIu8IeovREK38QnLH pjKqkntfnxpvDcdKwQNhu5gTKJSBTAhC5PwWcCsoFU+p5jq/62lmMdih31GPjieRbr8t BQ== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2qp9xu3xmq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:26 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIQ2K007388 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:26 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIQ84010001; Wed, 20 Feb 2019 20:18:26 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:25 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 29/39] KVM: x86/xen: evtchn unmask support Date: Wed, 20 Feb 2019 20:15:59 +0000 Message-Id: <20190220201609.28290-30-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=828 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ankur Arora Handle hypercall to unmasks event channel ports. A subtlety here is that we deliver an upcall if an actual unmask happened and the event channel was pending (and if the vcpu-wide evtchn_pending_sel was not already marked pending.) Co-developed-by: Joao Martins Signed-off-by: Ankur Arora Signed-off-by: Joao Martins --- arch/x86/kvm/xen.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 420e3ebb66bc..1988ed3866bf 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -2102,6 +2102,29 @@ static int shim_hcall_evtchn_send(struct kvm_xen *dom0, struct evtchn_send *snd) return 0; } +static int shim_hcall_evtchn_unmask(struct kvm_xen *dom0, int port) +{ + struct shared_info *s = HYPERVISOR_shared_info; + struct evtchnfd *evtchnfd; + struct vcpu_info *v; + + evtchnfd = idr_find(&dom0->port_to_evt, port); + if (!evtchnfd) + return -ENOENT; + + v = per_cpu(xen_vcpu, evtchnfd->vcpu); + + if (test_and_clear_bit(port, (unsigned long *) s->evtchn_mask) && + test_bit(port, (unsigned long *) s->evtchn_pending) && + !test_and_set_bit(port / BITS_PER_EVTCHN_WORD, + (unsigned long *) &v->evtchn_pending_sel)) { + kvm_xen_evtchn_2l_vcpu_set_pending(v); + return kvm_xen_evtchn_call_function(evtchnfd); + } + + return 0; +} + static int shim_hcall_evtchn(int op, void *p) { int ret; @@ -2151,6 +2174,13 @@ static int shim_hcall_evtchn(int op, void *p) un->port = evt.port; break; } + case EVTCHNOP_unmask: { + struct evtchn_unmask *unmask; + + unmask = (struct evtchn_unmask *) p; + ret = shim_hcall_evtchn_unmask(xen_shim, unmask->port); + break; + } case EVTCHNOP_send: { struct evtchn_send *send; From patchwork Wed Feb 20 20:16:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822649 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7AFAD13B5 for ; Wed, 20 Feb 2019 20:18:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 67C5F2E97D for ; Wed, 20 Feb 2019 20:18:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5B9D92EABE; Wed, 20 Feb 2019 20:18:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 09D302E97D for ; Wed, 20 Feb 2019 20:18:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728067AbfBTUSv (ORCPT ); Wed, 20 Feb 2019 15:18:51 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:53246 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728049AbfBTUSu (ORCPT ); Wed, 20 Feb 2019 15:18:50 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8ZDJ081724; Wed, 20 Feb 2019 20:18:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=SgHD9NVfwS2nZ8X3B3FNB8zCOkq9pVflU3V9h8Ufw4Y=; b=H4Q1jVpbdxB8niJnlfhNM+b5amup3s7H37wbFI/ZDHxpSsaYmKa3ISPDKi4+d/uywI30 zgHvqvHNlUtjcFM/7J1ozxcJobITDFgmeRJuBkmiH98AuGKExNSthwPQOorHk2OfSGmU sw1NNy0xYv6FJOQr09yCgp3QRQw0MFoMzrG58SMh40Fz0a9rh1x9NKvlBwfspxT/LVob 9cPrfPGZrQWA9F9VXDKB4BCtsJTM316KKRAzSTPINDv97b4poPUR0A/zG3NjtCP1NTxS drXjhXRxZRrDqmHAJnk5HpH+Kd//qhGz18JjNfX5i4JLlT4f5Z8KI5XW2lTXZ4y/9psH ug== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2130.oracle.com with ESMTP id 2qp9xu3xne-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:36 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIUR3019979 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:30 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKIUhI018949; Wed, 20 Feb 2019 20:18:30 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:29 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 30/39] KVM: x86/xen: add additional evtchn ops Date: Wed, 20 Feb 2019 20:16:00 +0000 Message-Id: <20190220201609.28290-31-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ankur Arora Add support for changing event channel affinity (EVTCHNOP_bind_vcpu) and closing an event (EVTCHNOP_close). We just piggy back on the functionality already implemented for guest event channels. Signed-off-by: Ankur Arora --- arch/x86/kvm/xen.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 1988ed3866bf..666dd6d1f5a3 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -2188,6 +2188,38 @@ static int shim_hcall_evtchn(int op, void *p) ret = shim_hcall_evtchn_send(xen_shim, send); break; } + case EVTCHNOP_bind_virq: { + struct evtchn_bind_virq *un; + + un = (struct evtchn_bind_virq *) p; + + evt.fd = -1; + evt.port = 0; + evt.type = XEN_EVTCHN_TYPE_VIRQ; + ret = kvm_xen_eventfd_assign(NULL, &xen_shim->port_to_evt, + &xen_shim->xen_lock, &evt); + un->port = evt.port; + break; + } + case EVTCHNOP_bind_vcpu: { + struct evtchn_bind_vcpu *bind_vcpu; + + bind_vcpu = (struct evtchn_bind_vcpu *) p; + + evt.port = bind_vcpu->port; + evt.vcpu = bind_vcpu->vcpu; + ret = kvm_xen_eventfd_update(NULL, &xen_shim->port_to_evt, + &xen_shim->xen_lock, &evt); + break; + } + case EVTCHNOP_close: { + struct evtchn_close *cls; + + cls = (struct evtchn_close *) p; + ret = kvm_xen_eventfd_deassign(NULL, &xen_shim->port_to_evt, + &xen_shim->xen_lock, cls->port); + break; + } default: ret = -EINVAL; break; From patchwork Wed Feb 20 20:16:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822651 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2135413B5 for ; Wed, 20 Feb 2019 20:18:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 063232E43F for ; Wed, 20 Feb 2019 20:18:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EB0FD2EE72; Wed, 20 Feb 2019 20:18:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F3FD12E43F for ; Wed, 20 Feb 2019 20:18:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728087AbfBTUSz (ORCPT ); Wed, 20 Feb 2019 15:18:55 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:34204 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726913AbfBTUSx (ORCPT ); Wed, 20 Feb 2019 15:18:53 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8e4A087617; Wed, 20 Feb 2019 20:18:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=iYIXYGyjPcBp7iJxj6+nN/3XpB73yvjLQ8ym2bw5tbg=; b=AYvhMbteyxJvtoiOMPGOViJorIFJPB6mXFoMYiKUbafbIQheKd7FpUBfsK4SlnKXyiS4 i4dFSFuY7cIw1GKBOEm9sf9RFMRkROvnDt0dccoeJJKfSdJmKlTdRenYpvYXbzadhLRO XJoPgpBVnCXwKDHGaUxBIKHYNWav9EDSrW+Kk+EbPiuGkkpmfx3u1KdxMpb6InCSZEP1 HO1+A/7b4Oj7cmvigaa9l1P8rejO4qUjtlJCieWDySZrQST0IMGZN66QUv9TFe0YeLIo x6JO/z8E4QVssuoeZNl2PAhHRFY8gzD1ehbG6wRy/E4tKv5hD+m3nwLSO1tGfjparKa5 Wg== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp2130.oracle.com with ESMTP id 2qp81ec3ea-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:35 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIYfZ020176 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:34 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIYDq010078; Wed, 20 Feb 2019 20:18:34 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:33 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org, Juergen Gross , Stefano Stabellini Subject: [PATCH RFC 31/39] xen-shim: introduce shim domain driver Date: Wed, 20 Feb 2019 20:16:01 +0000 Message-Id: <20190220201609.28290-32-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ankur Arora xen-shim.ko sets up and tears down state needed to support Xen backends. The underlying primitives that are exposed are interdomain event-channels and grant-table map/unmap/copy. We setup the following: * Initialize shared_info and vcpu_info pages, essentially setting up event-channel state. * Set up features (this allows xen_feature() to work) * Initialize event-channel subsystem (select event ops and related setup.) * Initialize xenbus and tear it down on module exit. This functionality would be used by the backend drivers (e.g. netback, scsiback, blkback etc) in order to drive guest I/O. Co-developed-by: Joao Martins Signed-off-by: Ankur Arora Signed-off-by: Joao Martins --- arch/x86/include/asm/kvm_host.h | 2 + arch/x86/kvm/Kconfig | 10 +++ arch/x86/kvm/Makefile | 1 + arch/x86/kvm/xen-shim.c | 107 +++++++++++++++++++++++++++++++ arch/x86/xen/enlighten.c | 45 +++++++++++++ drivers/xen/events/events_2l.c | 2 +- drivers/xen/events/events_base.c | 6 +- drivers/xen/features.c | 1 + drivers/xen/xenbus/xenbus_dev_frontend.c | 4 +- include/xen/xen.h | 5 ++ include/xen/xenbus.h | 3 + 11 files changed, 182 insertions(+), 4 deletions(-) create mode 100644 arch/x86/kvm/xen-shim.c diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 55609e919e14..6bdae8649d56 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -896,6 +896,8 @@ struct kvm_grant_table { /* Xen emulation context */ struct kvm_xen { u64 xen_hypercall; + +#define XEN_SHIM_DOMID 0 domid_t domid; gfn_t shinfo_addr; diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index 72fa955f4a15..47347df282dc 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -96,6 +96,16 @@ config KVM_MMU_AUDIT This option adds a R/W kVM module parameter 'mmu_audit', which allows auditing of KVM MMU events at runtime. +config XEN_SHIM + tristate "Xen hypercall emulation shim" + depends on KVM + depends on XEN + default m + help + Shim to support Xen hypercalls on non-Xen hosts. It intercepts grant + table and event channels hypercalls same way as Xen hypervisor. This is + useful for having Xen backend drivers work on KVM. + # OK, it's a little counter-intuitive to do this, but it puts it neatly under # the virtualization menu. source "drivers/vhost/Kconfig" diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile index c1eaabbd0a54..a96a96a002a7 100644 --- a/arch/x86/kvm/Makefile +++ b/arch/x86/kvm/Makefile @@ -18,3 +18,4 @@ kvm-amd-y += svm.o pmu_amd.o obj-$(CONFIG_KVM) += kvm.o obj-$(CONFIG_KVM_INTEL) += kvm-intel.o obj-$(CONFIG_KVM_AMD) += kvm-amd.o +obj-$(CONFIG_XEN_SHIM) += xen-shim.o diff --git a/arch/x86/kvm/xen-shim.c b/arch/x86/kvm/xen-shim.c new file mode 100644 index 000000000000..61fdceb63ec2 --- /dev/null +++ b/arch/x86/kvm/xen-shim.c @@ -0,0 +1,107 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2019 Oracle and/or its affiliates. All rights reserved. + * + * Xen hypercall emulation shim + */ + +#define pr_fmt(fmt) "KVM:" KBUILD_MODNAME ": " fmt + +#include + +#include +#include +#include +#include + +#define BITS_PER_EVTCHN_WORD (sizeof(xen_ulong_t)*8) + +static struct kvm_xen shim = { .domid = XEN_SHIM_DOMID }; + +static void shim_evtchn_setup(struct shared_info *s) +{ + int cpu; + + /* Point Xen's shared_info to the domain's sinfo page */ + HYPERVISOR_shared_info = s; + + /* Evtchns will be marked pending on allocation */ + memset(s->evtchn_pending, 0, sizeof(s->evtchn_pending)); + /* ... but we do mask all of -- dom0 expect it. */ + memset(s->evtchn_mask, 1, sizeof(s->evtchn_mask)); + + for_each_possible_cpu(cpu) { + struct vcpu_info *vcpu_info; + int i; + + /* Direct CPU mapping as far as dom0 is concerned */ + per_cpu(xen_vcpu_id, cpu) = cpu; + + vcpu_info = &per_cpu(xen_vcpu_info, cpu); + memset(vcpu_info, 0, sizeof(*vcpu_info)); + + vcpu_info->evtchn_upcall_mask = 0; + + vcpu_info->evtchn_upcall_pending = 0; + for (i = 0; i < BITS_PER_EVTCHN_WORD; i++) + clear_bit(i, &vcpu_info->evtchn_pending_sel); + + per_cpu(xen_vcpu, cpu) = vcpu_info; + } +} + +static int __init shim_register(void) +{ + struct shared_info *shinfo; + + shinfo = (struct shared_info *)get_zeroed_page(GFP_KERNEL); + if (!shinfo) { + pr_err("Failed to allocate shared_info page\n"); + return -ENOMEM; + } + shim.shinfo = shinfo; + + idr_init(&shim.port_to_evt); + mutex_init(&shim.xen_lock); + + kvm_xen_register_lcall(&shim); + + /* We can handle hypercalls after this point */ + xen_shim_domain = 1; + + shim_evtchn_setup(shim.shinfo); + + xen_setup_features(); + + xen_init_IRQ(); + + xenbus_init(); + + return 0; +} + +static int __init shim_init(void) +{ + if (xen_domain()) + return -ENODEV; + + return shim_register(); +} + +static void __exit shim_exit(void) +{ + xenbus_deinit(); + xen_shim_domain = 0; + + kvm_xen_unregister_lcall(); + HYPERVISOR_shared_info = NULL; + free_page((unsigned long) shim.shinfo); + shim.shinfo = NULL; +} + +module_init(shim_init); +module_exit(shim_exit) + +MODULE_AUTHOR("Ankur Arora ," + "Joao Martins "); +MODULE_LICENSE("GPL"); diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index b36a10e6b5d7..8d9e93b6eb09 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -57,6 +57,9 @@ EXPORT_PER_CPU_SYMBOL(xen_vcpu_info); enum xen_domain_type xen_domain_type = XEN_NATIVE; EXPORT_SYMBOL_GPL(xen_domain_type); +int xen_shim_domain; +EXPORT_SYMBOL_GPL(xen_shim_domain); + unsigned long *machine_to_phys_mapping = (void *)MACH2PHYS_VIRT_START; EXPORT_SYMBOL(machine_to_phys_mapping); unsigned long machine_to_phys_nr; @@ -349,3 +352,45 @@ void xen_arch_unregister_cpu(int num) } EXPORT_SYMBOL(xen_arch_unregister_cpu); #endif + +static struct module *find_module_shim(void) +{ + static const char name[] = "xen_shim"; + struct module *module; + + mutex_lock(&module_mutex); + module = find_module(name); + mutex_unlock(&module_mutex); + + return module; +} + +bool xen_shim_domain_get(void) +{ + struct module *shim; + + if (!xen_shim_domain()) + return false; + + shim = find_module_shim(); + if (!shim) + return false; + + return try_module_get(shim); +} +EXPORT_SYMBOL(xen_shim_domain_get); + +void xen_shim_domain_put(void) +{ + struct module *shim; + + if (!xen_shim_domain()) + return; + + shim = find_module_shim(); + if (!shim) + return; + + module_put(shim); +} +EXPORT_SYMBOL(xen_shim_domain_put); diff --git a/drivers/xen/events/events_2l.c b/drivers/xen/events/events_2l.c index b5acf4b09971..f08d13a033c1 100644 --- a/drivers/xen/events/events_2l.c +++ b/drivers/xen/events/events_2l.c @@ -89,7 +89,7 @@ static void evtchn_2l_unmask(unsigned port) unsigned int cpu = get_cpu(); int do_hypercall = 0, evtchn_pending = 0; - BUG_ON(!irqs_disabled()); + WARN_ON(!irqs_disabled()); if (unlikely((cpu != cpu_from_evtchn(port)))) do_hypercall = 1; diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c index 117e76b2f939..a2087287c3b6 100644 --- a/drivers/xen/events/events_base.c +++ b/drivers/xen/events/events_base.c @@ -1665,7 +1665,7 @@ void xen_callback_vector(void) {} static bool fifo_events = true; module_param(fifo_events, bool, 0); -void __init xen_init_IRQ(void) +void xen_init_IRQ(void) { int ret = -EINVAL; unsigned int evtchn; @@ -1683,6 +1683,9 @@ void __init xen_init_IRQ(void) for (evtchn = 0; evtchn < xen_evtchn_nr_channels(); evtchn++) mask_evtchn(evtchn); + if (xen_shim_domain()) + return; + pirq_needs_eoi = pirq_needs_eoi_flag; #ifdef CONFIG_X86 @@ -1714,3 +1717,4 @@ void __init xen_init_IRQ(void) } #endif } +EXPORT_SYMBOL_GPL(xen_init_IRQ); diff --git a/drivers/xen/features.c b/drivers/xen/features.c index d7d34fdfc993..1518c3b6f004 100644 --- a/drivers/xen/features.c +++ b/drivers/xen/features.c @@ -31,3 +31,4 @@ void xen_setup_features(void) xen_features[i * 32 + j] = !!(fi.submap & 1< diff --git a/include/xen/xenbus.h b/include/xen/xenbus.h index 869c816d5f8c..d2789e7d2055 100644 --- a/include/xen/xenbus.h +++ b/include/xen/xenbus.h @@ -233,4 +233,7 @@ extern const struct file_operations xen_xenbus_fops; extern struct xenstore_domain_interface *xen_store_interface; extern int xen_store_evtchn; +int xenbus_init(void); +void xenbus_deinit(void); + #endif /* _XEN_XENBUS_H */ From patchwork Wed Feb 20 20:16:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822671 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8237A17E9 for ; Wed, 20 Feb 2019 20:20:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6C2692F17A for ; Wed, 20 Feb 2019 20:20:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CDDEF2F0E8; Wed, 20 Feb 2019 20:20:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C3DB12F112 for ; Wed, 20 Feb 2019 20:20:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728012AbfBTUT6 (ORCPT ); Wed, 20 Feb 2019 15:19:58 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:54318 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725989AbfBTUT5 (ORCPT ); Wed, 20 Feb 2019 15:19:57 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8qDU081939; Wed, 20 Feb 2019 20:18:40 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=NTxOcurqJlQlCOmdT9Cg4RcY8w77Z669tNvu7Rl2B6U=; b=fEydcBYmCx/tjgu15N7dbOIE0EU5eFtRpT5j0YjNrZqPj9jXx1LM7r10/PRHAjlttaSR 5An38nBBVf8/StS0LCjeqK+02rd6CDpvXSotOc0vKrdsrzPi5ksKquLfk4+BhcUCNwYr /RVsNmR5Tq31CMgw7SfXxaW3yro+vFz6A/f+NhhL31hlk8gd0tMFWwRj9mLtnjUi3VDl /dLFHluxBLAdd1zj8lMnNwT8wykSPFpRyiRWqMQxFZWai/WoYGuOjfyWAnNx3UkyYFhu epq4+VLpxoogmD/SfiZeCkdpeSAxU/f0D/nrRi3odnGNr9Qnws6MJCzDDjIlr88+80pm 0w== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2qp9xu3xnn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:39 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIcSo025549 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:39 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIbUF012376; Wed, 20 Feb 2019 20:18:38 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:37 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org, Juergen Gross , Stefano Stabellini Subject: [PATCH RFC 32/39] xen/balloon: xen_shim_domain() support Date: Wed, 20 Feb 2019 20:16:02 +0000 Message-Id: <20190220201609.28290-33-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Xen ballooning uses hollow struct pages (with the underlying PFNs being populated/unpopulated via hypercalls) which are used by the grant logic to map grants from other domains. For purposes of a KVM based xen-shim, this model is not useful -- mapping is unnecessary since all guest memory is already mapped in the KVM host. The simplest option is to just translate grant references to GPAs (essentially a get_page() on the appropriate GPA.) This patch provides an alternate balloon allocation mechanism where in the allocation path we just provide a constant struct page (corresponding to page 0.) This allows the calling code -- which does a page_to_pfn() on the returned struct page -- to remain unchanged before doing the grant operation (which in this case would fill in the real struct page.) Co-developed-by: Ankur Arora Signed-off-by: Joao Martins Signed-off-by: Ankur Arora --- arch/x86/kvm/xen-shim.c | 31 +++++++++++++++++++++++++++++++ drivers/xen/balloon.c | 15 ++++++++++++++- include/xen/balloon.h | 7 +++++++ 3 files changed, 52 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/xen-shim.c b/arch/x86/kvm/xen-shim.c index 61fdceb63ec2..4086d92a4bfb 100644 --- a/arch/x86/kvm/xen-shim.c +++ b/arch/x86/kvm/xen-shim.c @@ -13,11 +13,40 @@ #include #include #include +#include #define BITS_PER_EVTCHN_WORD (sizeof(xen_ulong_t)*8) static struct kvm_xen shim = { .domid = XEN_SHIM_DOMID }; +static int shim_alloc_pages(int nr_pages, struct page **pages) +{ + int i; + + /* + * We provide page 0 instead of NULL because we'll effectively + * do the inverse operation while deriving the pfn to pass to + * xen for mapping. + */ + for (i = 0; i < nr_pages; i++) + pages[i] = pfn_to_page(0); + + return 0; +} + +static void shim_free_pages(int nr_pages, struct page **pages) +{ + int i; + + for (i = 0; i < nr_pages; i++) + pages[i] = NULL; +} + +static struct xen_balloon_ops shim_balloon_ops = { + .alloc_pages = shim_alloc_pages, + .free_pages = shim_free_pages, +}; + static void shim_evtchn_setup(struct shared_info *s) { int cpu; @@ -65,6 +94,7 @@ static int __init shim_register(void) mutex_init(&shim.xen_lock); kvm_xen_register_lcall(&shim); + xen_balloon_ops = &shim_balloon_ops; /* We can handle hypercalls after this point */ xen_shim_domain = 1; @@ -94,6 +124,7 @@ static void __exit shim_exit(void) xen_shim_domain = 0; kvm_xen_unregister_lcall(); + xen_balloon_ops = NULL; HYPERVISOR_shared_info = NULL; free_page((unsigned long) shim.shinfo); shim.shinfo = NULL; diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c index ceb5048de9a7..00375fa6c122 100644 --- a/drivers/xen/balloon.c +++ b/drivers/xen/balloon.c @@ -138,7 +138,7 @@ enum bp_state { static DEFINE_MUTEX(balloon_mutex); -struct balloon_stats balloon_stats; +struct balloon_stats balloon_stats __read_mostly; EXPORT_SYMBOL_GPL(balloon_stats); /* We increase/decrease in batches which fit in a page */ @@ -158,6 +158,9 @@ static DECLARE_DELAYED_WORK(balloon_worker, balloon_process); #define GFP_BALLOON \ (GFP_HIGHUSER | __GFP_NOWARN | __GFP_NORETRY | __GFP_NOMEMALLOC) +struct xen_balloon_ops *xen_balloon_ops; +EXPORT_SYMBOL(xen_balloon_ops); + /* balloon_append: add the given page to the balloon. */ static void __balloon_append(struct page *page) { @@ -589,6 +592,11 @@ int alloc_xenballooned_pages(int nr_pages, struct page **pages) struct page *page; int ret; + if (xen_shim_domain() && xen_balloon_ops) + return xen_balloon_ops->alloc_pages(nr_pages, pages); + + WARN_ON_ONCE(xen_shim_domain()); + mutex_lock(&balloon_mutex); balloon_stats.target_unpopulated += nr_pages; @@ -634,6 +642,11 @@ void free_xenballooned_pages(int nr_pages, struct page **pages) { int i; + if (xen_shim_domain() && xen_balloon_ops) + return xen_balloon_ops->free_pages(nr_pages, pages); + + WARN_ON_ONCE(xen_shim_domain()); + mutex_lock(&balloon_mutex); for (i = 0; i < nr_pages; i++) { diff --git a/include/xen/balloon.h b/include/xen/balloon.h index 4914b93a23f2..9ba6a7e91d5e 100644 --- a/include/xen/balloon.h +++ b/include/xen/balloon.h @@ -22,6 +22,13 @@ struct balloon_stats { extern struct balloon_stats balloon_stats; +struct xen_balloon_ops { + int (*alloc_pages)(int nr_pages, struct page **pages); + void (*free_pages)(int nr_pages, struct page **pages); +}; + +extern struct xen_balloon_ops *xen_balloon_ops; + void balloon_set_new_target(unsigned long target); int alloc_xenballooned_pages(int nr_pages, struct page **pages); From patchwork Wed Feb 20 20:16:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822667 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 97C5C17E9 for ; Wed, 20 Feb 2019 20:20:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6FFD02F198 for ; Wed, 20 Feb 2019 20:20:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 05DDE2F099; Wed, 20 Feb 2019 20:20:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 699B82F099 for ; Wed, 20 Feb 2019 20:20:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728242AbfBTUT6 (ORCPT ); Wed, 20 Feb 2019 15:19:58 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:36402 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728006AbfBTUSs (ORCPT ); Wed, 20 Feb 2019 15:18:48 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8q7r087687; Wed, 20 Feb 2019 20:18:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=Q0vNiEu8C1xVftpLfrPYy6je6m9dd4hOUd4QG/a02TY=; b=3d3E4nAra5kn7VEKjXGgHnWDCQ+AoVXbep3lcjzOtPqjQNiphlNJ33AzrOvGcnJPGa9N a1Gbz0HKCfxI12qxxuynWPVPIAnvJdCN6Br5n26ypxXHVFaTk37eNjty8K4hGzZZOrIf ar8KoBAw3dZbdIZC/W2zEOoAAFI2UVoPX81Ssfy+83vLPNfsfM+jcypDcIQTfd7I7Ztq jvyGT0EzU/GQwoOI9fAnjHHnDiu6slHzL59kfnQq10ArrVMNGDOh7W5vdxExjDgy4hZL dZoNTjd747JtABjMse/G8ac5a2B48IcExQE6TK06LfBCVfriqUMW9+S3WwxXiiOu7n8Q PA== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2qpb5rktvx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:41 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIeNX025682 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:41 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIeWm010196; Wed, 20 Feb 2019 20:18:40 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:40 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Juergen Gross , Stefano Stabellini Subject: [PATCH RFC 33/39] xen/grant-table: xen_shim_domain() support Date: Wed, 20 Feb 2019 20:16:03 +0000 Message-Id: <20190220201609.28290-34-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=952 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With xen-shim, allocate_xenballooned_pages() only allocates a place-holder page (pfn 0) expecting a subsequent map_grant_ref to fix it up. However, this means that, until the grant operation (GNTTABOP_map_grant_ref) provides a valid page, we cannot set PagePrivate or save any state. This patch elides the setting of that state if xen_shim_domain(). In addition, gnttab_map_refs() now fills in the appropriate page returned from the grant operation. Signed-off-by: Joao Martins --- drivers/xen/grant-table.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c index 7ea6fb6a2e5d..ab05b70d98bb 100644 --- a/drivers/xen/grant-table.c +++ b/drivers/xen/grant-table.c @@ -804,7 +804,7 @@ int gnttab_alloc_pages(int nr_pages, struct page **pages) int ret; ret = alloc_xenballooned_pages(nr_pages, pages); - if (ret < 0) + if (ret < 0 || xen_shim_domain()) return ret; ret = gnttab_pages_set_private(nr_pages, pages); @@ -1045,6 +1045,11 @@ int gnttab_map_refs(struct gnttab_map_grant_ref *map_ops, { struct xen_page_foreign *foreign; + if (xen_shim_domain()) { + pages[i] = virt_to_page(map_ops[i].host_addr); + continue; + } + SetPageForeign(pages[i]); foreign = xen_page_foreign(pages[i]); foreign->domid = map_ops[i].dom; @@ -1085,8 +1090,10 @@ int gnttab_unmap_refs(struct gnttab_unmap_grant_ref *unmap_ops, if (ret) return ret; - for (i = 0; i < count; i++) - ClearPageForeign(pages[i]); + for (i = 0; i < count; i++) { + if (!xen_shim_domain()) + ClearPageForeign(pages[i]); + } return clear_foreign_p2m_mapping(unmap_ops, kunmap_ops, pages, count); } @@ -1113,7 +1120,7 @@ static void __gnttab_unmap_refs_async(struct gntab_unmap_queue_data* item) int pc; for (pc = 0; pc < item->count; pc++) { - if (page_count(item->pages[pc]) > 1) { + if (page_count(item->pages[pc]) > 1 && !xen_shim_domain()) { unsigned long delay = GNTTAB_UNMAP_REFS_DELAY * (item->age + 1); schedule_delayed_work(&item->gnttab_work, msecs_to_jiffies(delay)); From patchwork Wed Feb 20 20:16:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822655 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 10EB117E9 for ; Wed, 20 Feb 2019 20:19:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EE91A2F01F for ; Wed, 20 Feb 2019 20:19:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DD31B2F021; Wed, 20 Feb 2019 20:19:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 854162F01F for ; Wed, 20 Feb 2019 20:19:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727338AbfBTUTV (ORCPT ); Wed, 20 Feb 2019 15:19:21 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:34694 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726613AbfBTUTV (ORCPT ); Wed, 20 Feb 2019 15:19:21 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8e4C087617; Wed, 20 Feb 2019 20:18:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=vahAq3kKddmSkfjcBh53XGqFljJf1oYQypzdVmOyw0Y=; b=RyJv7v5K27ef8U03dFrDBbsDxmmO4aIGNBOtI81x2cmx/7TVwwNKoBlZd5brgZA4MTLO m17CZsghn/1QSkk+la1M9cfqqGmLGGpo720hasLol/xOPGIFe9inMW28EwUHD/4ppYVk NNq67Usy+Gff+2eC/GMNXir30+4EO+ycoRKAZ79gidUi0Ki+ZqzQwqx/qR1fRInSiOAC dQN11OuM585EIR+JDDh2WWkVDeqc5zEsV5thOjgpvCDR99ogNQi5TRS0aSaWITeO+ILF QkJplgMyVc1q0fPDEKHkAvvxt0p6Y04IstT2mZbtrGm1F7qkmO4AD0ZX0hSc7uZCirBA uw== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp2130.oracle.com with ESMTP id 2qp81ec3fc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:50 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIidk020563 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:44 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKIhYb015978; Wed, 20 Feb 2019 20:18:43 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:43 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Juergen Gross , Stefano Stabellini Subject: [PATCH RFC 34/39] xen/gntdev: xen_shim_domain() support Date: Wed, 20 Feb 2019 20:16:04 +0000 Message-Id: <20190220201609.28290-35-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=758 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ankur Arora GNTTABOP_map_grant_ref treats host_addr as an OUT parameter for xen_shim_domaim(). Accordingly it's updated in struct gnttab_unmap_grant_ref before it gets used via GNTTABOP_unmap_grant_ref. Co-developed-by: Joao Martins Signed-off-by: Ankur Arora Signed-off-by: Joao Martins --- drivers/xen/gntdev.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index 5efc5eee9544..8540a51f7597 100644 --- a/drivers/xen/gntdev.c +++ b/drivers/xen/gntdev.c @@ -351,6 +351,8 @@ int gntdev_map_grant_pages(struct gntdev_grant_map *map) } map->unmap_ops[i].handle = map->map_ops[i].handle; + if (xen_shim_domain()) + map->unmap_ops[i].host_addr = map->map_ops[i].host_addr; if (use_ptemod) map->kunmap_ops[i].handle = map->kmap_ops[i].handle; #ifdef CONFIG_XEN_GRANT_DMA_ALLOC @@ -1122,7 +1124,9 @@ static int gntdev_mmap(struct file *flip, struct vm_area_struct *vma) (map->flags & GNTMAP_readonly)) goto out_unlock_put; } else { - map->flags = GNTMAP_host_map; + map->flags = 0; + if (!xen_shim_domain()) + map->flags = GNTMAP_host_map; if (!(vma->vm_flags & VM_WRITE)) map->flags |= GNTMAP_readonly; } @@ -1207,7 +1211,7 @@ static int __init gntdev_init(void) { int err; - if (!xen_domain()) + if (!xen_domain() && !xen_shim_domain_get()) return -ENODEV; use_ptemod = !xen_feature(XENFEAT_auto_translated_physmap); @@ -1215,6 +1219,7 @@ static int __init gntdev_init(void) err = misc_register(&gntdev_miscdev); if (err != 0) { pr_err("Could not register gntdev device\n"); + xen_shim_domain_put(); return err; } return 0; @@ -1223,6 +1228,7 @@ static int __init gntdev_init(void) static void __exit gntdev_exit(void) { misc_deregister(&gntdev_miscdev); + xen_shim_domain_put(); } module_init(gntdev_init); From patchwork Wed Feb 20 20:16:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822663 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C1D1D17E9 for ; Wed, 20 Feb 2019 20:19:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD9682F01F for ; Wed, 20 Feb 2019 20:19:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A1FA32F021; Wed, 20 Feb 2019 20:19:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17E702F01F for ; Wed, 20 Feb 2019 20:19:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728113AbfBTUTD (ORCPT ); Wed, 20 Feb 2019 15:19:03 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:34258 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726878AbfBTUSy (ORCPT ); Wed, 20 Feb 2019 15:18:54 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8VMo087440; Wed, 20 Feb 2019 20:18:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=nEoaALBLcsNsxHnJFDNeJyCL6A1vZHWL3Ev7hoUXOR8=; b=L7GnQZKkBuiZR6S84VaVdAO/PMf0RxkH2Ijy80O0lmcjcqbkn3ABj626knGXcZaDVUIF BDZWhrJlcEoHOpGhhTwFiolKxI76K9L/u8TuTNtXXnddgsuZUbCMZDdVop3NkH9ofrmZ 6s56Y77lgri2vfCxVvz0a9Dye5kPNQzpcJ+7/517VfOVDd4yOxmFEaxPUbYVY3C4pc94 xGbzKEQoKeN4W693bFRat7J9Eqs6C4ISUQgwDNHL6AFYDk7o8HXyVPGhtXBDTQZnhsqA eZJDc/2CZtc9oB5sx3xpxHM+YLbrEsYV8Z5ID7iqZkzZaXRt0LFOc7rBUc+ouprS+exQ Lg== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp2130.oracle.com with ESMTP id 2qp81ec3f4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:47 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIkdD020684 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:46 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKIkfj019110; Wed, 20 Feb 2019 20:18:46 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:45 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Juergen Gross , Stefano Stabellini Subject: [PATCH RFC 35/39] xen/xenbus: xen_shim_domain() support Date: Wed, 20 Feb 2019 20:16:05 +0000 Message-Id: <20190220201609.28290-36-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ankur Arora Fixup the gnttab unmap_ops (and other data structures) to handle host_addr as an OUT parameter from the call to GNTTABOP_map_grant_ref. Also, allow xenstored to be hosted in XS_LOCAL mode for xen_shim_domain() -- this means that it does not need to acquire xenstore evtchn and pfn externally. Co-developed-by: Joao Martins Signed-off-by: Ankur Arora Signed-off-by: Joao Martins --- drivers/xen/xenbus/xenbus_client.c | 23 +++++++++++++++++------ drivers/xen/xenbus/xenbus_dev_backend.c | 4 ++-- drivers/xen/xenbus/xenbus_dev_frontend.c | 4 ++-- drivers/xen/xenbus/xenbus_probe.c | 8 ++++---- 4 files changed, 25 insertions(+), 14 deletions(-) diff --git a/drivers/xen/xenbus/xenbus_client.c b/drivers/xen/xenbus/xenbus_client.c index ada1c9aa6525..b22149a789d4 100644 --- a/drivers/xen/xenbus/xenbus_client.c +++ b/drivers/xen/xenbus/xenbus_client.c @@ -487,8 +487,11 @@ static int __xenbus_map_ring(struct xenbus_device *dev, "mapping in shared page %d from domain %d", gnt_refs[i], dev->otherend_id); goto fail; - } else + } else { handles[i] = map[i].handle; + if (xen_shim_domain()) + addrs[i] = map[i].host_addr; + } } return GNTST_okay; @@ -498,7 +501,8 @@ static int __xenbus_map_ring(struct xenbus_device *dev, if (handles[i] != INVALID_GRANT_HANDLE) { memset(&unmap[j], 0, sizeof(unmap[j])); gnttab_set_unmap_op(&unmap[j], (phys_addr_t)addrs[i], - GNTMAP_host_map, handles[i]); + !xen_shim_domain()?GNTMAP_host_map:0, + handles[i]); j++; } } @@ -546,7 +550,7 @@ static int xenbus_map_ring_valloc_hvm(struct xenbus_device *dev, void **vaddr) { struct xenbus_map_node *node; - int err; + int i, err; void *addr; bool leaked = false; struct map_ring_valloc_hvm info = { @@ -572,9 +576,16 @@ static int xenbus_map_ring_valloc_hvm(struct xenbus_device *dev, &info); err = __xenbus_map_ring(dev, gnt_ref, nr_grefs, node->handles, - info.phys_addrs, GNTMAP_host_map, &leaked); + info.phys_addrs, + !xen_shim_domain() ? GNTMAP_host_map : 0, + &leaked); node->nr_handles = nr_grefs; + if (xen_shim_domain()) { + for (i = 0; i < nr_grefs; i++) + node->hvm.pages[i] = virt_to_page(info.phys_addrs[i]); + } + if (err) goto out_free_ballooned_pages; @@ -882,7 +893,7 @@ int xenbus_unmap_ring(struct xenbus_device *dev, for (i = 0; i < nr_handles; i++) gnttab_set_unmap_op(&unmap[i], vaddrs[i], - GNTMAP_host_map, handles[i]); + !xen_shim_domain()?GNTMAP_host_map:0, handles[i]); if (HYPERVISOR_grant_table_op(GNTTABOP_unmap_grant_ref, unmap, i)) BUG(); @@ -926,7 +937,7 @@ static const struct xenbus_ring_ops ring_ops_hvm = { .unmap = xenbus_unmap_ring_vfree_hvm, }; -void __init xenbus_ring_ops_init(void) +void xenbus_ring_ops_init(void) { #ifdef CONFIG_XEN_PV if (!xen_feature(XENFEAT_auto_translated_physmap)) diff --git a/drivers/xen/xenbus/xenbus_dev_backend.c b/drivers/xen/xenbus/xenbus_dev_backend.c index edba5fecde4d..b605c87bff76 100644 --- a/drivers/xen/xenbus/xenbus_dev_backend.c +++ b/drivers/xen/xenbus/xenbus_dev_backend.c @@ -119,11 +119,11 @@ static struct miscdevice xenbus_backend_dev = { .fops = &xenbus_backend_fops, }; -static int __init xenbus_backend_init(void) +int xenbus_backend_init(void) { int err; - if (!xen_initial_domain()) + if (!xen_initial_domain() && !xen_shim_domain()) return -ENODEV; err = misc_register(&xenbus_backend_dev); diff --git a/drivers/xen/xenbus/xenbus_dev_frontend.c b/drivers/xen/xenbus/xenbus_dev_frontend.c index a4080d04a01c..c6fca6cca6c8 100644 --- a/drivers/xen/xenbus/xenbus_dev_frontend.c +++ b/drivers/xen/xenbus/xenbus_dev_frontend.c @@ -680,11 +680,11 @@ static struct miscdevice xenbus_dev = { .fops = &xen_xenbus_fops, }; -static int __init xenbus_frontend_init(void) +static int xenbus_frontend_init(void) { int err; - if (!xen_domain()) + if (!xen_domain() && !xen_shim_domain()) return -ENODEV; err = misc_register(&xenbus_dev); diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index 2e0ed46b05e7..bbc405cd01ef 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -693,7 +693,7 @@ EXPORT_SYMBOL_GPL(xenbus_probe); static int __init xenbus_probe_initcall(void) { - if (!xen_domain()) + if (!xen_domain() && !xen_shim_domain()) return -ENODEV; if (xen_initial_domain() || xen_hvm_domain()) @@ -790,7 +790,7 @@ int xenbus_init(void) uint64_t v = 0; xen_store_domain_type = XS_UNKNOWN; - if (!xen_domain()) + if (!xen_domain() && !xen_shim_domain()) return -ENODEV; xenbus_ring_ops_init(); @@ -799,7 +799,7 @@ int xenbus_init(void) xen_store_domain_type = XS_PV; if (xen_hvm_domain()) xen_store_domain_type = XS_HVM; - if (xen_hvm_domain() && xen_initial_domain()) + if ((xen_hvm_domain() && xen_initial_domain()) || xen_shim_domain()) xen_store_domain_type = XS_LOCAL; if (xen_pv_domain() && !xen_start_info->store_evtchn) xen_store_domain_type = XS_LOCAL; @@ -863,7 +863,7 @@ postcore_initcall(xenbus_init); void xenbus_deinit(void) { - if (!xen_domain()) + if (!xen_domain() && !xen_shim_domain()) return; #ifdef CONFIG_XEN_COMPAT_XENFS From patchwork Wed Feb 20 20:16:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822673 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B15661399 for ; Wed, 20 Feb 2019 20:20:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9E4512EE89 for ; Wed, 20 Feb 2019 20:20:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 915A12F04E; Wed, 20 Feb 2019 20:20:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6FD4E2F072 for ; Wed, 20 Feb 2019 20:20:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727849AbfBTUUV (ORCPT ); Wed, 20 Feb 2019 15:20:21 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:54704 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727717AbfBTUUU (ORCPT ); Wed, 20 Feb 2019 15:20:20 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK98rL082386; Wed, 20 Feb 2019 20:18:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=jPor7XctkGZCsqvXc242v28Xv2yJ7KWqcRDiRETXPRI=; b=F8H07j44UR1tdSp9IX24yuzkWPkmOz7N1HefA6nBH2ZGm0d/i5epC0lBYdUHGt3ufw1F CE6uD+ubucHMlnQwjhdTI9MHKcgb0aDCiZW3NG9vS1FE3WFxi6wE8BITz8fjwQU1unbH /mFjA/OD2BIyscqlvV1nfjG4Ck28RNljs339RqQUaKcZomkhtl8WthEwf7NZYGjS/0jw viANzo+l5MwQKLEZMrQSza7IzGGqPiUn6FEfFkW06HIPzObL3axqkEaPeYmr43zooBBB gTJF1PuXjEwAqRpSFuFvN6WiMG6Xp6QUND9uoRzEDo01qD1nDt6NiePgL/ybzle3DnfK rQ== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2qp9xu3xp7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:50 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKInTQ008317 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:49 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKInBo012477; Wed, 20 Feb 2019 20:18:49 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:48 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Juergen Gross , Stefano Stabellini Subject: [PATCH RFC 36/39] drivers/xen: xen_shim_domain() support Date: Wed, 20 Feb 2019 20:16:06 +0000 Message-Id: <20190220201609.28290-37-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=994 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Enable /dev/xen/{gntdev,evtchn} and /proc/xen/ for xen_shim_domain(). These interfaces will be used by xenstored to initialize its event channel port and the kva used to communicate with the xenbus driver. Signed-off-by: Joao Martins --- drivers/xen/evtchn.c | 4 +++- drivers/xen/privcmd.c | 5 ++++- drivers/xen/xenbus/xenbus_xs.c | 2 +- drivers/xen/xenfs/super.c | 7 ++++--- drivers/xen/xenfs/xensyms.c | 4 ++++ 5 files changed, 16 insertions(+), 6 deletions(-) diff --git a/drivers/xen/evtchn.c b/drivers/xen/evtchn.c index 6d1a5e58968f..367390eb5883 100644 --- a/drivers/xen/evtchn.c +++ b/drivers/xen/evtchn.c @@ -708,13 +708,14 @@ static int __init evtchn_init(void) { int err; - if (!xen_domain()) + if (!xen_domain() && !xen_shim_domain_get()) return -ENODEV; /* Create '/dev/xen/evtchn'. */ err = misc_register(&evtchn_miscdev); if (err != 0) { pr_err("Could not register /dev/xen/evtchn\n"); + xen_shim_domain_put(); return err; } @@ -726,6 +727,7 @@ static int __init evtchn_init(void) static void __exit evtchn_cleanup(void) { misc_deregister(&evtchn_miscdev); + xen_shim_domain_put(); } module_init(evtchn_init); diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c index b24ddac1604b..a063ad5c8260 100644 --- a/drivers/xen/privcmd.c +++ b/drivers/xen/privcmd.c @@ -999,12 +999,13 @@ static int __init privcmd_init(void) { int err; - if (!xen_domain()) + if (!xen_domain() && !xen_shim_domain_get()) return -ENODEV; err = misc_register(&privcmd_dev); if (err != 0) { pr_err("Could not register Xen privcmd device\n"); + xen_shim_domain_put(); return err; } @@ -1012,6 +1013,7 @@ static int __init privcmd_init(void) if (err != 0) { pr_err("Could not register Xen hypercall-buf device\n"); misc_deregister(&privcmd_dev); + xen_shim_domain_put(); return err; } @@ -1022,6 +1024,7 @@ static void __exit privcmd_exit(void) { misc_deregister(&privcmd_dev); misc_deregister(&xen_privcmdbuf_dev); + xen_shim_domain_put(); } module_init(privcmd_init); diff --git a/drivers/xen/xenbus/xenbus_xs.c b/drivers/xen/xenbus/xenbus_xs.c index bd6db3703972..062c0a5fa415 100644 --- a/drivers/xen/xenbus/xenbus_xs.c +++ b/drivers/xen/xenbus/xenbus_xs.c @@ -735,7 +735,7 @@ static void xs_reset_watches(void) { int err; - if (!xen_hvm_domain() || xen_initial_domain()) + if (!xen_hvm_domain() || xen_initial_domain() || xen_shim_domain()) return; if (xen_strict_xenbus_quirk()) diff --git a/drivers/xen/xenfs/super.c b/drivers/xen/xenfs/super.c index 71ddfb4cf61c..2ef326645880 100644 --- a/drivers/xen/xenfs/super.c +++ b/drivers/xen/xenfs/super.c @@ -64,7 +64,8 @@ static int xenfs_fill_super(struct super_block *sb, void *data, int silent) }; return simple_fill_super(sb, XENFS_SUPER_MAGIC, - xen_initial_domain() ? xenfs_init_files : xenfs_files); + xen_initial_domain() || xen_shim_domain() ? + xenfs_init_files : xenfs_files); } static struct dentry *xenfs_mount(struct file_system_type *fs_type, @@ -84,7 +85,7 @@ MODULE_ALIAS_FS("xenfs"); static int __init xenfs_init(void) { - if (xen_domain()) + if (xen_domain() || xen_shim_domain()) return register_filesystem(&xenfs_type); return 0; @@ -92,7 +93,7 @@ static int __init xenfs_init(void) static void __exit xenfs_exit(void) { - if (xen_domain()) + if (xen_domain() || xen_shim_domain()) unregister_filesystem(&xenfs_type); } diff --git a/drivers/xen/xenfs/xensyms.c b/drivers/xen/xenfs/xensyms.c index c6c73a33c44d..79bccb53d344 100644 --- a/drivers/xen/xenfs/xensyms.c +++ b/drivers/xen/xenfs/xensyms.c @@ -8,6 +8,7 @@ #include #include #include +#include #include "xenfs.h" @@ -114,6 +115,9 @@ static int xensyms_open(struct inode *inode, struct file *file) struct xensyms *xs; int ret; + if (xen_shim_domain()) + return -EINVAL; + ret = seq_open_private(file, &xensyms_seq_ops, sizeof(struct xensyms)); if (ret) From patchwork Wed Feb 20 20:16:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822665 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DBA1D922 for ; Wed, 20 Feb 2019 20:20:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C28372F01F for ; Wed, 20 Feb 2019 20:20:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B4A142F068; Wed, 20 Feb 2019 20:20:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F8F82F01F for ; Wed, 20 Feb 2019 20:19:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726913AbfBTUTt (ORCPT ); Wed, 20 Feb 2019 15:19:49 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:36676 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728088AbfBTUTC (ORCPT ); Wed, 20 Feb 2019 15:19:02 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8X0E087279; Wed, 20 Feb 2019 20:18:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=laSsJtG6e1YDxSFuHMs6Owotdsw5hc2nXFNlpfsUZKg=; b=byZj9a47q2n/BwPkweTGZr/L+IzWXktfnCkaa6NZ8O/u6MhAitnLTVVtso3MyhvndhX7 e7qETlpjAcaVPoxXw3zLVOZbCfv3RftZ57K9ByYd/fLxPSrdGWENst96VeJUDYJ9K2lL 43q3jfKzXkqIJnHP9yS3XONUwgOuIZrmrxDDiR5OOJP1LlyTUc7JxwVso3nYxGONuvKj OyfhmJR9QNbIqGsjvYq9FDOgHK6hCnW0vR1aSFes+ENFxCX5wEoo3glfLIsF84SzyJqh yQh11jsEE4k/ak6ecBWWBTd2YQn/XN+9muPvRd2ZjfNIzzSnTBGZBxC8ieWx88QlpEV3 5w== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2qpb5rktxg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:57 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIp34030809 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:51 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIpvb010277; Wed, 20 Feb 2019 20:18:51 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:51 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Wei Liu , Paul Durrant Subject: [PATCH RFC 37/39] xen-netback: xen_shim_domain() support Date: Wed, 20 Feb 2019 20:16:07 +0000 Message-Id: <20190220201609.28290-38-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP GNTTABOP_map_grant_ref treats host_address as an OUT parameter for xen_shim_domain(). After doing gnttab_map_refs, we check each map op struct and replace the page passed with the page returned by the hypercall. Note that mmap_pages is just a placeholder when netback runs in xen_shim_domain() mode. Signed-off-by: Joao Martins --- drivers/net/xen-netback/netback.c | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c index 80aae3a32c2a..a5dd6de6d3e6 100644 --- a/drivers/net/xen-netback/netback.c +++ b/drivers/net/xen-netback/netback.c @@ -324,6 +324,9 @@ struct xenvif_tx_cb { #define XENVIF_TX_CB(skb) ((struct xenvif_tx_cb *)(skb)->cb) +/* Selects host map only if on native Xen */ +#define GNTTAB_host_map (xen_shim_domain() ? 0 : GNTMAP_host_map) + static inline void xenvif_tx_create_map_op(struct xenvif_queue *queue, u16 pending_idx, struct xen_netif_tx_request *txp, @@ -332,9 +335,12 @@ static inline void xenvif_tx_create_map_op(struct xenvif_queue *queue, { queue->pages_to_map[mop-queue->tx_map_ops] = queue->mmap_pages[pending_idx]; gnttab_set_map_op(mop, idx_to_kaddr(queue, pending_idx), - GNTMAP_host_map | GNTMAP_readonly, + GNTTAB_host_map | GNTMAP_readonly, txp->gref, queue->vif->domid); + if (xen_shim_domain()) + mop->host_addr = 0; + memcpy(&queue->pending_tx_info[pending_idx].req, txp, sizeof(*txp)); queue->pending_tx_info[pending_idx].extra_count = extra_count; @@ -405,6 +411,12 @@ static struct gnttab_map_grant_ref *xenvif_get_requests(struct xenvif_queue *que return gop; } +static inline void xenvif_grant_replace_page(struct xenvif_queue *queue, + u16 pending_idx, u32 mapping_idx) +{ + queue->mmap_pages[pending_idx] = queue->pages_to_map[mapping_idx]; +} + static inline void xenvif_grant_handle_set(struct xenvif_queue *queue, u16 pending_idx, grant_handle_t handle) @@ -481,6 +493,10 @@ static int xenvif_tx_check_gop(struct xenvif_queue *queue, xenvif_grant_handle_set(queue, pending_idx, gop_map->handle); + + if (!err && xen_shim_domain()) + xenvif_grant_replace_page(queue, pending_idx, + gop_map - queue->tx_map_ops); /* Had a previous error? Invalidate this fragment. */ if (unlikely(err)) { xenvif_idx_unmap(queue, pending_idx); @@ -1268,7 +1284,7 @@ static inline void xenvif_tx_dealloc_action(struct xenvif_queue *queue) queue->mmap_pages[pending_idx]; gnttab_set_unmap_op(gop, idx_to_kaddr(queue, pending_idx), - GNTMAP_host_map, + GNTTAB_host_map, queue->grant_tx_handle[pending_idx]); xenvif_grant_handle_reset(queue, pending_idx); ++gop; @@ -1394,7 +1410,7 @@ void xenvif_idx_unmap(struct xenvif_queue *queue, u16 pending_idx) gnttab_set_unmap_op(&tx_unmap_op, idx_to_kaddr(queue, pending_idx), - GNTMAP_host_map, + GNTTAB_host_map, queue->grant_tx_handle[pending_idx]); xenvif_grant_handle_reset(queue, pending_idx); @@ -1622,7 +1638,7 @@ static int __init netback_init(void) { int rc = 0; - if (!xen_domain()) + if (!xen_domain() && !xen_shim_domain_get()) return -ENODEV; /* Allow as many queues as there are CPUs but max. 8 if user has not @@ -1663,6 +1679,7 @@ static void __exit netback_fini(void) debugfs_remove_recursive(xen_netback_dbg_root); #endif /* CONFIG_DEBUG_FS */ xenvif_xenbus_fini(); + xen_shim_domain_put(); } module_exit(netback_fini); From patchwork Wed Feb 20 20:16:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822661 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 693C3922 for ; Wed, 20 Feb 2019 20:19:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5668D2F01F for ; Wed, 20 Feb 2019 20:19:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4A49F2F021; Wed, 20 Feb 2019 20:19:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C2ACF2F01F for ; Wed, 20 Feb 2019 20:19:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728123AbfBTUTE (ORCPT ); Wed, 20 Feb 2019 15:19:04 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:36674 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728084AbfBTUTC (ORCPT ); Wed, 20 Feb 2019 15:19:02 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8X0D087279; Wed, 20 Feb 2019 20:18:55 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=0lg03zJLVaBGqwjLBLKs78/Wx7pcPDjqBK/bhI6FzTQ=; b=x4g9//AfL0MpMsu69Y6hq7AoBMds9YX5yL1OhI5AxZL/LQXRuWBUi0oC4WuGIcn1XAo1 UZns63sifFX/x0nuectoMd9FXjqfvTC8ozgAr0Q68jx9AFgpHOnECikWTYfRBePPglHG 0C197uIL8dwHMxhbal2FpqPbVplGnTrBs//LyMSWvmZZe9mdUNSTEKaQQwM/ayVgYgo8 E0oCP3+cQ/zk05FJF+QnUwbeAetFfXBnj/6/lwHrO1Yob5VTAG8PXdlOXAYgGQD0gT6k tIVFF0B2imyafuvg3fytEkSIOc/oWqE5udrUbeFcZ9U7C2e8mgm4kaZ/pbS+YiTKOqN4 0Q== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2qpb5rktxa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:55 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIsgM030964 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:54 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIs1D010292; Wed, 20 Feb 2019 20:18:54 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:53 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Konrad Rzeszutek Wilk , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= Subject: [PATCH RFC 38/39] xen-blkback: xen_shim_domain() support Date: Wed, 20 Feb 2019 20:16:08 +0000 Message-Id: <20190220201609.28290-39-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=810 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With xen_shim_domain() enabled, the struct pages used for grant mappings are only valid in the interval between GNTTABOP_map_grant_ref and GNTTABOP_unmap_grant_ref. Ensure that we do not cache the struct page outside that duration. Also, update the struct page for the segment after GNTTABOP_map_grant_ref, for the subsequent tracking or when doing IO. In addition, we disable persistent grants for this configuration, since the map/unmap overhead is fairly small (in the fast path, page lookup, get_page() and put_page().) This reduces the memory usage in blkback/blkfront. Co-developed-by: Ankur Arora Signed-off-by: Joao Martins Signed-off-by: Ankur Arora --- drivers/block/xen-blkback/blkback.c | 19 ++++++++++++++++--- drivers/block/xen-blkback/xenbus.c | 5 +++-- 2 files changed, 19 insertions(+), 5 deletions(-) diff --git a/drivers/block/xen-blkback/blkback.c b/drivers/block/xen-blkback/blkback.c index d51d88be88e1..20c15e6787b1 100644 --- a/drivers/block/xen-blkback/blkback.c +++ b/drivers/block/xen-blkback/blkback.c @@ -167,6 +167,15 @@ static inline void put_free_pages(struct xen_blkif_ring *ring, struct page **pag unsigned long flags; int i; + /* + * We don't own the struct page after unmap has been called. + * Reallocation is cheap anyway for the shim domain case, so free it. + */ + if (xen_shim_domain()) { + gnttab_free_pages(num, page); + return; + } + spin_lock_irqsave(&ring->free_pages_lock, flags); for (i = 0; i < num; i++) list_add(&page[i]->lru, &ring->free_pages); @@ -825,7 +834,7 @@ static int xen_blkbk_map(struct xen_blkif_ring *ring, */ again: for (i = map_until; i < num; i++) { - uint32_t flags; + uint32_t flags = 0; if (use_persistent_gnts) { persistent_gnt = get_persistent_gnt( @@ -846,7 +855,8 @@ static int xen_blkbk_map(struct xen_blkif_ring *ring, addr = vaddr(pages[i]->page); pages_to_gnt[segs_to_map] = pages[i]->page; pages[i]->persistent_gnt = NULL; - flags = GNTMAP_host_map; + if (!xen_shim_domain()) + flags = GNTMAP_host_map; if (!use_persistent_gnts && ro) flags |= GNTMAP_readonly; gnttab_set_map_op(&map[segs_to_map++], addr, @@ -880,6 +890,8 @@ static int xen_blkbk_map(struct xen_blkif_ring *ring, goto next; } pages[seg_idx]->handle = map[new_map_idx].handle; + if (xen_shim_domain()) + pages[seg_idx]->page = pages_to_gnt[new_map_idx]; } else { continue; } @@ -1478,7 +1490,7 @@ static int __init xen_blkif_init(void) { int rc = 0; - if (!xen_domain()) + if (!xen_domain() && !xen_shim_domain_get()) return -ENODEV; if (xen_blkif_max_ring_order > XENBUS_MAX_RING_GRANT_ORDER) { @@ -1508,6 +1520,7 @@ static void __exit xen_blkif_exit(void) { xen_blkif_interface_exit(); xen_blkif_xenbus_exit(); + xen_shim_domain_put(); } module_exit(xen_blkif_exit); diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c index 424e2efebe85..11be0c052b77 100644 --- a/drivers/block/xen-blkback/xenbus.c +++ b/drivers/block/xen-blkback/xenbus.c @@ -871,7 +871,8 @@ static void connect(struct backend_info *be) xen_blkbk_barrier(xbt, be, be->blkif->vbd.flush_support); - err = xenbus_printf(xbt, dev->nodename, "feature-persistent", "%u", 1); + err = xenbus_printf(xbt, dev->nodename, "feature-persistent", "%u", + !xen_shim_domain()); if (err) { xenbus_dev_fatal(dev, err, "writing %s/feature-persistent", dev->nodename); @@ -1059,7 +1060,7 @@ static int connect_ring(struct backend_info *be) } pers_grants = xenbus_read_unsigned(dev->otherend, "feature-persistent", 0); - be->blkif->vbd.feature_gnt_persistent = pers_grants; + be->blkif->vbd.feature_gnt_persistent = pers_grants && !xen_shim_domain(); be->blkif->vbd.overflow_max_grants = 0; /* From patchwork Wed Feb 20 20:16:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 10822653 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ED69F1805 for ; Wed, 20 Feb 2019 20:19:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D91012E698 for ; Wed, 20 Feb 2019 20:19:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CBE072EE97; Wed, 20 Feb 2019 20:19:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5D29B2E698 for ; Wed, 20 Feb 2019 20:19:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728184AbfBTUTP (ORCPT ); Wed, 20 Feb 2019 15:19:15 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:34592 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728193AbfBTUTN (ORCPT ); Wed, 20 Feb 2019 15:19:13 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1KK8lad087668; Wed, 20 Feb 2019 20:18:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=FLMolSeoCWdbxtw77CaFuuJsBsJh/M3jVztlkbWRZPc=; b=5MChGTBTyedjta2OKtVVtqGVHWi2hrR0XbsE6nCrI+qGtbyLVQNeLpU+asie5DxyKi1K U+spM7BcYJe/YtXAadQbuczyROFG0IGr5n8J96FPM2jF0VlfBwE3gMtKl7/y2eiFiCFU sMZWTPeQj4pGIRDC2D0wDyC8KQBWeCMHyZDg16bnrrUlmketGwEgI3fWu8erYAzekv+N MIbdxz9/GCI5GOsR8nPVQBU6XRdMbpUT4PXcz5OjKxH3Tyef2QmXmsHxjS8Kv2AfzHbq hI4sENM+0q2KzrkbQji6Aav7f3fNnM5mBHFOlPlPf9JBTHNQXGc4m970Rr9saEFPZqmt iA== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2qp81ec3gd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:58 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1KKIvTU031200 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Feb 2019 20:18:57 GMT Received: from abhmp0022.oracle.com (abhmp0022.oracle.com [141.146.116.28]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1KKIvZh019192; Wed, 20 Feb 2019 20:18:57 GMT Received: from paddy.lan (/94.61.137.133) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 12:18:56 -0800 From: Joao Martins To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ankur Arora , Boris Ostrovsky , Joao Martins , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH RFC 39/39] KVM: x86: declare Xen HVM Dom0 capability Date: Wed, 20 Feb 2019 20:16:09 +0000 Message-Id: <20190220201609.28290-40-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190220201609.28290-1-joao.m.martins@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=699 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902200138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add new capability for domid, interdomain/unbound event channel types and grant table support in hypervisor. This would be used to drive Xen kernel backends. Co-developed-by: Ankur Arora Signed-off-by: Joao Martins Signed-off-by: Ankur Arora --- Documentation/virtual/kvm/api.txt | 10 ++++++++++ arch/x86/kvm/x86.c | 4 ++++ include/uapi/linux/kvm.h | 3 +++ 3 files changed, 17 insertions(+) diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt index 36d9386415fa..311dcded5e28 100644 --- a/Documentation/virtual/kvm/api.txt +++ b/Documentation/virtual/kvm/api.txt @@ -5046,3 +5046,13 @@ This capability indicates KVM's support for the event channel offload. Implies support for KVM_IRQ_ROUTING_XEN_EVTCHN irq routing, and for attribute KVM_XEN_ATTR_TYPE_EVTCHN in KVM_XEN_HVM_GET_ATTR or KVM_XEN_HVM_SET_ATTR. + +8.24 KVM_CAP_XEN_HVM_DOM0 + +Architectures: x86 + +This capability indicates support for assigning domid and handling kernel +backends in the hypervisor. Also implies that attributes +KVM_XEN_ATTR_TYPE_DOMID, KVM_XEN_ATTR_TYPE_GNTTAB are supported. For the +existing KVM_XEN_ATTR_TYPE_EVTCHN attribute, it indicates support for +interdomain and unbound event channels. diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index cb95f7f8bed9..e8c3494b10cb 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -71,6 +71,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include "trace.h" @@ -3049,6 +3050,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_ADJUST_CLOCK: r = KVM_CLOCK_TSC_STABLE; break; + case KVM_CAP_XEN_HVM_DOM0: + r = xen_shim_domain(); + break; case KVM_CAP_X86_DISABLE_EXITS: r |= KVM_X86_DISABLE_EXITS_HLT | KVM_X86_DISABLE_EXITS_PAUSE; if(kvm_can_mwait_in_guest()) diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 74d877792dfa..d817a7bbf507 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1005,6 +1005,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_HYPERV_CPUID 167 #define KVM_CAP_XEN_HVM_GUEST 168 #define KVM_CAP_XEN_HVM_EVTCHN 169 +#define KVM_CAP_XEN_HVM_DOM0 170 #ifdef KVM_CAP_IRQ_ROUTING @@ -1485,6 +1486,7 @@ struct kvm_xen_hvm_attr { #define XEN_EVTCHN_TYPE_VIRQ 0 #define XEN_EVTCHN_TYPE_IPI 1 +/* Available with KVM_CAP_XEN_HVM_DOM0 */ #define XEN_EVTCHN_TYPE_INTERDOM 2 #define XEN_EVTCHN_TYPE_UNBOUND 3 __u32 type; @@ -1536,6 +1538,7 @@ struct kvm_xen_hvm_attr { #define KVM_XEN_ATTR_TYPE_VCPU_RUNSTATE 0x3 /* Available with KVM_CAP_XEN_HVM_EVTCHN */ #define KVM_XEN_ATTR_TYPE_EVTCHN 0x4 +/* Available with KVM_CAP_XEN_HVM_DOM0 */ #define KVM_XEN_ATTR_TYPE_DOMID 0x5 #define KVM_XEN_ATTR_TYPE_GNTTAB 0x6