From patchwork Fri Jan 10 16:04:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 11327803 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1144192A for ; Fri, 10 Jan 2020 16:05:50 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E1B78206ED for ; Fri, 10 Jan 2020 16:05:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="OgzQ8rDX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E1B78206ED Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ipwmE-00005P-9U; Fri, 10 Jan 2020 16:04:46 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ipwmD-000056-Nz for xen-devel@lists.xenproject.org; Fri, 10 Jan 2020 16:04:45 +0000 X-Inumbo-ID: e380fd98-33c2-11ea-a985-bc764e2007e4 Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id e380fd98-33c2-11ea-a985-bc764e2007e4; Fri, 10 Jan 2020 16:04:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1578672273; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7HSVWQWUuSbIKADLhBHupdJ3JAQZlKjh+SU49yxI7Ww=; b=OgzQ8rDXLh60DHnanX8IVrY5WuoKx4BNX0rDwqSSg5d30aF6C1669iWn vb9rNaGNeZnvV0CYvpRT1Co6iHRcliOhtE9M5l5y18gSifrYD8eenbV2D v4NXjjxKYvLTsQlTRUd9h9pPJX9xhT7mt6ee8Eb7y1A28Evhondsrhn8U Y=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa3.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: w7RoXuVrBZXc2YFnqn8ZI0/rjEGB/aWaX6YqivnJRgVoHFNiwhLaeOBYcQyLJjRKAaIQhaIcWo /vrUlbIHG+sde9QXHXoh7AuTHqEzZdosE4WmFaMQbx0YNPaavhHxddbs1zX6LJd+0yb3aVC1yH vsSnUrHGE0X8qWdqxikHNGgvOnQScVO111YR9Vfepm5mZkKNk+O//aL/DKa6gyefY3x9KfO5uA mtpmNgbKy3PyyWMlCFlbnUKjDhLc2Et4Am8brncTEQ5Mj29n6IKWS490PIbqbxSzxVU0Nv8PZv tvM= X-SBRS: 2.7 X-MesageID: 10736127 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,417,1571716800"; d="scan'208";a="10736127" From: Roger Pau Monne To: Date: Fri, 10 Jan 2020 17:04:04 +0100 Message-ID: <20200110160404.15573-4-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200110160404.15573-1-roger.pau@citrix.com> References: <20200110160404.15573-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v2 3/3] x86/tlb: use Xen L0 assisted TLB flush when available X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Use Xen's L0 HVMOP_flush_tlbs hypercall in order to perform flushes. This greatly increases the performance of TLB flushes when running with a high amount of vCPUs as a Xen guest, and is specially important when running in shim mode. The following figures are from a PV guest running `make -j32 xen` in shim mode with 32 vCPUs. Using x2APIC and ALLBUT shorthand: real 4m35.973s user 4m35.110s sys 36m24.117s Using L0 assisted flush: real 1m14.206s user 4m31.835s sys 5m45.013s The implementation adds a new hook to hypervisor_ops so other enlightenments can also implement such assisted flush just by filling the hook. Note that the Xen implementation completely ignores the dirty CPU mask and the linear address passed in, and always performs a global TLB flush on all vCPUs. Signed-off-by: Roger Pau Monné --- Changes since v1: - Add a L0 assisted hook to hypervisor ops. --- xen/arch/x86/guest/hypervisor.c | 9 +++++++++ xen/arch/x86/guest/xen/xen.c | 6 ++++++ xen/arch/x86/smp.c | 6 ++++++ xen/include/asm-x86/guest/hypervisor.h | 13 +++++++++++++ 4 files changed, 34 insertions(+) diff --git a/xen/arch/x86/guest/hypervisor.c b/xen/arch/x86/guest/hypervisor.c index 4f27b98740..c793ba51c2 100644 --- a/xen/arch/x86/guest/hypervisor.c +++ b/xen/arch/x86/guest/hypervisor.c @@ -18,6 +18,7 @@ * * Copyright (c) 2019 Microsoft. */ +#include #include #include @@ -64,6 +65,14 @@ void hypervisor_resume(void) ops->resume(); } +int hypervisor_flush_tlb(const cpumask_t *mask, const void *va) +{ + if ( ops && ops->flush_tlb ) + return ops->flush_tlb(mask, va); + + return -ENOSYS; +} + /* * Local variables: * mode: C diff --git a/xen/arch/x86/guest/xen/xen.c b/xen/arch/x86/guest/xen/xen.c index 6dbc5f953f..47af773aca 100644 --- a/xen/arch/x86/guest/xen/xen.c +++ b/xen/arch/x86/guest/xen/xen.c @@ -310,11 +310,17 @@ static void resume(void) pv_console_init(); } +static int flush_tlb(const cpumask_t *mask, const void *va) +{ + return xen_hypercall_hvm_op(HVMOP_flush_tlbs, NULL); +} + static const struct hypervisor_ops ops = { .name = "Xen", .setup = setup, .ap_setup = ap_setup, .resume = resume, + .flush_tlb = flush_tlb, }; const struct hypervisor_ops *__init xg_probe(void) diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c index 6510dd84ab..b3cb1ba90f 100644 --- a/xen/arch/x86/smp.c +++ b/xen/arch/x86/smp.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -262,6 +263,11 @@ void flush_area_mask(const cpumask_t *mask, const void *va, unsigned int flags) if ( (flags & ~FLUSH_ORDER_MASK) && !cpumask_subset(mask, cpumask_of(cpu)) ) { + if ( cpu_has_hypervisor && + !(flags & ~(FLUSH_TLB | FLUSH_TLB_GLOBAL | FLUSH_VA_VALID)) && + !hypervisor_flush_tlb(mask, va) ) + return; + spin_lock(&flush_lock); cpumask_and(&flush_cpumask, mask, &cpu_online_map); cpumask_clear_cpu(cpu, &flush_cpumask); diff --git a/xen/include/asm-x86/guest/hypervisor.h b/xen/include/asm-x86/guest/hypervisor.h index 392f4b90ae..b7f1969796 100644 --- a/xen/include/asm-x86/guest/hypervisor.h +++ b/xen/include/asm-x86/guest/hypervisor.h @@ -28,6 +28,8 @@ struct hypervisor_ops { void (*ap_setup)(void); /* Resume from suspension */ void (*resume)(void); + /* L0 assisted TLB flush */ + int (*flush_tlb)(const cpumask_t *mask, const void *va); }; #ifdef CONFIG_GUEST @@ -36,9 +38,16 @@ const char *hypervisor_probe(void); void hypervisor_setup(void); void hypervisor_ap_setup(void); void hypervisor_resume(void); +/* + * L0 assisted TLB flush. + * mask: cpumask of the dirty vCPUs that should be flushed. + * va: linear address to flush, or NULL for global flushes. + */ +int hypervisor_flush_tlb(const cpumask_t *mask, const void *va); #else +#include #include #include @@ -46,6 +55,10 @@ static inline const char *hypervisor_probe(void) { return NULL; } static inline void hypervisor_setup(void) { ASSERT_UNREACHABLE(); } static inline void hypervisor_ap_setup(void) { ASSERT_UNREACHABLE(); } static inline void hypervisor_resume(void) { ASSERT_UNREACHABLE(); } +static inline int hypervisor_flush_tlb(const cpumask_t *mask, const void *va) +{ + return -ENOSYS; +} #endif /* CONFIG_GUEST */