From patchwork Fri Apr 17 15:50:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11495445 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9970781 for ; Fri, 17 Apr 2020 15:50:50 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 75D7420857 for ; Fri, 17 Apr 2020 15:50:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="hRtN2I0O" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 75D7420857 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jPTFy-0007WE-6x; Fri, 17 Apr 2020 15:50:18 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jPTFw-0007W9-W9 for xen-devel@lists.xenproject.org; Fri, 17 Apr 2020 15:50:17 +0000 X-Inumbo-ID: 1e35ab82-80c3-11ea-8d2d-12813bfff9fa Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 1e35ab82-80c3-11ea-8d2d-12813bfff9fa; Fri, 17 Apr 2020 15:50:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1587138611; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=n+kJcIVbiUyfNa/vyQETwoQaqxnGj4WaK08bU0mpih8=; b=hRtN2I0OIE8y+Zi/dpQjUqLCsSAU9I3WoxwYgvRFatSvSMPtdD+0K+Bg N2cuCa65YRccGg5LHrXop7ylPBa9p06jsYP3IYeEM6WA85qFJNI1JlhYS 0obBJgsPf2iGiIosKz/FH2LH/y49jZcbm/4MePNTk2nmJf0ymNcjK0fnm I=; Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa6.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: 8Vcdiq6a3gEzu1kd9D3SXRhu8odtLdwfaNLMVV4pcfP+CBcV0GsgdPkLh86Q4M6llxFbBZSO8i lBGg6e9Rc2fwNZLdoFh7bY9V2GnlMyuixjajzPIZ5rHqb/Upld0OdrEezDv+HMUatSCfCnj3Hf iMDN6RY/uWDJ7ezfGmRAXh9BPOAMqmoM79JyJSkAeGJJSk3epCxXb7G2a1Pwv5/pTNf+8Bkwez Dku+pWBL//AfyNwjr7IN86o6UbvTK1+8Te73vE2rAcj1cUQ8mlMYuQTziAIcKTei9o2Z8HCWpP dPI= X-SBRS: 2.7 X-MesageID: 16253793 X-Ironport-Server: esa6.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.72,395,1580792400"; d="scan'208";a="16253793" From: Andrew Cooper To: Xen-devel Subject: [PATCH 1/3] x86/pv: Options to disable and/or compile out 32bit PV support Date: Fri, 17 Apr 2020 16:50:02 +0100 Message-ID: <20200417155004.16806-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200417155004.16806-1-andrew.cooper3@citrix.com> References: <20200417155004.16806-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" This is the start of some performance and security-hardening improvements, based on the fact that 32bit PV guests are few and far between these days. Ring1 is full or architectural corner cases, such as counting as supervisor from a paging point of view. This accounts for a substantial performance hit on processors from the last 8 years (adjusting SMEP/SMAP on every privilege transition), and the gap is only going to get bigger with new hardware features. Signed-off-by: Andrew Cooper Reviewed-by: Roger Pau Monné --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné There is a series I can't quite post yet which wants to conditionally turn opt_pv32 off, which is why I've put it straight in in an int8_t form rather than a straight boolean form. --- docs/misc/xen-command-line.pandoc | 12 +++++++++++- xen/arch/x86/Kconfig | 16 ++++++++++++++++ xen/arch/x86/pv/domain.c | 35 +++++++++++++++++++++++++++++++++++ xen/arch/x86/setup.c | 9 +++++++-- xen/include/asm-x86/pv/domain.h | 6 ++++++ 5 files changed, 75 insertions(+), 3 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index acd0b3d994..ee12b0f53f 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -1694,7 +1694,17 @@ The following resources are available: CDP, one COS will corespond two CBMs other than one with CAT, due to the sum of CBMs is fixed, that means actual `cos_max` in use will automatically reduce to half when CDP is enabled. - + +### pv + = List of [ 32= ] + + Applicability: x86 + +Controls for aspects of PV guest support. + +* The `32` boolean controls whether 32bit PV guests can be created. It + defaults to `true`, and is ignored when `CONFIG_PV32` is compiled out. + ### pv-linear-pt (x86) > `= ` diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig index 8149362bde..4c52197de3 100644 --- a/xen/arch/x86/Kconfig +++ b/xen/arch/x86/Kconfig @@ -49,6 +49,22 @@ config PV If unsure, say Y. +config PV32 + bool "Support for 32bit PV guests" + depends on PV + default y + ---help--- + The 32bit PV ABI uses Ring1, an area of the x86 architecture which + was deprecated and mostly removed in the AMD64 spec. As a result, + it occasionally conflicts with newer x86 hardware features, causing + overheads for Xen to maintain backwards compatibility. + + People may wish to disable 32bit PV guests for attack surface + reduction, or performance reasons. Backwards compatibility can be + provided via the PV Shim mechanism. + + If unsure, say Y. + config PV_LINEAR_PT bool "Support for PV linear pagetables" depends on PV diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c index 70fae43965..47a0db082f 100644 --- a/xen/arch/x86/pv/domain.c +++ b/xen/arch/x86/pv/domain.c @@ -16,6 +16,39 @@ #include #include +#ifdef CONFIG_PV32 +int8_t __read_mostly opt_pv32 = -1; +#endif + +static int parse_pv(const char *s) +{ + const char *ss; + int val, rc = 0; + + do { + ss = strchr(s, ','); + if ( !ss ) + ss = strchr(s, '\0'); + + if ( (val = parse_boolean("32", s, ss)) >= 0 ) + { +#ifdef CONFIG_PV32 + opt_pv32 = val; +#else + printk(XENLOG_INFO + "CONFIG_PV32 disabled - ignoring 'pv=32' setting\n"); +#endif + } + else + rc = -EINVAL; + + s = ss + 1; + } while ( *ss ); + + return rc; +} +custom_param("pv", parse_pv); + static __read_mostly enum { PCID_OFF, PCID_ALL, @@ -174,6 +207,8 @@ int switch_compat(struct domain *d) BUILD_BUG_ON(offsetof(struct shared_info, vcpu_info) != 0); + if ( !opt_pv32 ) + return -EOPNOTSUPP; if ( is_hvm_domain(d) || domain_tot_pages(d) != 0 ) return -EACCES; if ( is_pv_32bit_domain(d) ) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index 885919d5c3..c50aefb2de 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -53,6 +53,7 @@ #include #include #include +#include /* opt_nosmp: If true, secondary processors are ignored. */ static bool __initdata opt_nosmp; @@ -1875,8 +1876,12 @@ void arch_get_xen_caps(xen_capabilities_info_t *info) { snprintf(s, sizeof(s), "xen-%d.%d-x86_64 ", major, minor); safe_strcat(*info, s); - snprintf(s, sizeof(s), "xen-%d.%d-x86_32p ", major, minor); - safe_strcat(*info, s); + + if ( opt_pv32 ) + { + snprintf(s, sizeof(s), "xen-%d.%d-x86_32p ", major, minor); + safe_strcat(*info, s); + } } if ( hvm_enabled ) { diff --git a/xen/include/asm-x86/pv/domain.h b/xen/include/asm-x86/pv/domain.h index 7a69bfb303..df9716ff26 100644 --- a/xen/include/asm-x86/pv/domain.h +++ b/xen/include/asm-x86/pv/domain.h @@ -23,6 +23,12 @@ #include +#ifdef CONFIG_PV32 +extern int8_t opt_pv32; +#else +# define opt_pv32 false +#endif + /* * PCID values for the address spaces of 64-bit pv domains: * From patchwork Fri Apr 17 15:50:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11495449 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A555181 for ; Fri, 17 Apr 2020 15:51:18 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7F58520936 for ; Fri, 17 Apr 2020 15:51:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="OCq2Kqcw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7F58520936 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jPTG3-0007Wi-JL; Fri, 17 Apr 2020 15:50:23 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jPTG1-0007WZ-WD for xen-devel@lists.xenproject.org; Fri, 17 Apr 2020 15:50:22 +0000 X-Inumbo-ID: 1f61c945-80c3-11ea-8d2d-12813bfff9fa Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 1f61c945-80c3-11ea-8d2d-12813bfff9fa; Fri, 17 Apr 2020 15:50:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1587138612; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZhK0+BzJnp1VoPrHaLr5cGNVpfHqgyvx1ITJ+8zW0CY=; b=OCq2KqcwQhkfBU0cDmGVB0GA+B51j0jHk2AAGtCru15GqK6R6V3ET5cI kpoSzCd7pMlZHvu+XV6mEMeDXmzUcRdvNdD7ZYMCisHAfwz2/vUiIcEuW v5QZqGuDMqUGr1hTRISJV2DNWTejoDNJwCrmCCEn+oqyuPEBUyIJ7M75t g=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: W60PJzhwI65JY1GYUh57Ly621HUTbcLiEz5oy0VTNXg7PnHPDHnYNirQiboUboOIulASbJDdr1 poaZgku4gp0ROTF6jflE5I2WFFCLUwM0Ya2F4PWcxEXzKi4y+zYd1smgUspOHsm8J95tdeSzlp +OKmDFAywofaNFoW08QuQFuap1MD0xHI8w7Ovt3Nyi32fFbRBZOBNswxmR3iVUdI/bbXFjPUQW uU+YPouzi5QerAjeF09mIejvZPFIUMO+zrzu6EmUG3W+qmJBDCJqDEy+RGjkDI+W0TUnAwD2oY 0pQ= X-SBRS: 2.7 X-MesageID: 16168814 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.72,395,1580792400"; d="scan'208";a="16168814" From: Andrew Cooper To: Xen-devel Subject: [PATCH 2/3] x86/pv: Short-circuit is_pv_{32, 64}bit_domain() in !CONFIG_PV32 builds Date: Fri, 17 Apr 2020 16:50:03 +0100 Message-ID: <20200417155004.16806-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200417155004.16806-1-andrew.cooper3@citrix.com> References: <20200417155004.16806-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" ... and move arch.is_32bit_pv into the pv union while at it. Bloat-o-meter reports the following net savings with some notable differences highlighted: add/remove: 4/6 grow/shrink: 5/76 up/down: 1955/-18792 (-16837) Function old new delta ... pv_vcpu_initialise 411 158 -253 guest_cpuid 1837 1584 -253 pv_hypercall 579 297 -282 check_descriptor 427 130 -297 _get_page_type 5915 5202 -713 arch_get_info_guest 2225 1195 -1030 context_switch 3831 2635 -1196 dom0_construct_pv 10284 8939 -1345 arch_set_info_guest 5564 3267 -2297 Total: Before=3079563, After=3062726, chg -0.55% In principle, DOMAIN_is_32bit_pv should be based on CONFIG_PV32, but the assembly code is going to need further untangling before that becomes easy to do. For now, use CONFIG_PV as missed accidentally by c/s ec651bd2460 "x86: make entry point code build when !CONFIG_PV". Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/domctl.c | 4 ++-- xen/arch/x86/pv/domain.c | 6 +++--- xen/arch/x86/pv/hypercall.c | 2 ++ xen/arch/x86/x86_64/asm-offsets.c | 4 +++- xen/include/asm-x86/domain.h | 4 ++-- xen/include/xen/sched.h | 15 +++++++++++++-- 6 files changed, 25 insertions(+), 10 deletions(-) diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index add70126b9..3822dd7fd1 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -576,8 +576,8 @@ long arch_do_domctl( ret = -EOPNOTSUPP; else if ( is_pv_domain(d) ) { - if ( ((domctl->u.address_size.size == 64) && !d->arch.is_32bit_pv) || - ((domctl->u.address_size.size == 32) && d->arch.is_32bit_pv) ) + if ( ((domctl->u.address_size.size == 64) && !d->arch.pv.is_32bit) || + ((domctl->u.address_size.size == 32) && d->arch.pv.is_32bit) ) ret = 0; else if ( domctl->u.address_size.size == 32 ) ret = switch_compat(d); diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c index 47a0db082f..e0977bfbd7 100644 --- a/xen/arch/x86/pv/domain.c +++ b/xen/arch/x86/pv/domain.c @@ -215,7 +215,7 @@ int switch_compat(struct domain *d) return 0; d->arch.has_32bit_shinfo = 1; - d->arch.is_32bit_pv = 1; + d->arch.pv.is_32bit = 1; for_each_vcpu( d, v ) { @@ -235,7 +235,7 @@ int switch_compat(struct domain *d) return 0; undo_and_fail: - d->arch.is_32bit_pv = d->arch.has_32bit_shinfo = 0; + d->arch.pv.is_32bit = d->arch.has_32bit_shinfo = 0; for_each_vcpu( d, v ) { free_compat_arg_xlat(v); @@ -358,7 +358,7 @@ int pv_domain_initialise(struct domain *d) d->arch.ctxt_switch = &pv_csw; /* 64-bit PV guest by default. */ - d->arch.is_32bit_pv = d->arch.has_32bit_shinfo = 0; + d->arch.pv.is_32bit = d->arch.has_32bit_shinfo = 0; d->arch.pv.xpti = is_hardware_domain(d) ? opt_xpti_hwdom : opt_xpti_domu; diff --git a/xen/arch/x86/pv/hypercall.c b/xen/arch/x86/pv/hypercall.c index 17ddf9ea1f..32d90a543f 100644 --- a/xen/arch/x86/pv/hypercall.c +++ b/xen/arch/x86/pv/hypercall.c @@ -302,6 +302,7 @@ void pv_ring3_init_hypercall_page(void *p) } } +#ifdef CONFIG_PV32 void pv_ring1_init_hypercall_page(void *p) { unsigned int i; @@ -329,6 +330,7 @@ void pv_ring1_init_hypercall_page(void *p) *(u8 *)(p+ 7) = 0xc3; /* ret */ } } +#endif /* * Local variables: diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c index 500df7a3e7..9f66a69be7 100644 --- a/xen/arch/x86/x86_64/asm-offsets.c +++ b/xen/arch/x86/x86_64/asm-offsets.c @@ -98,8 +98,10 @@ void __dummy__(void) OFFSET(VCPU_nsvm_hap_enabled, struct vcpu, arch.hvm.nvcpu.u.nsvm.ns_hap_enabled); BLANK(); - OFFSET(DOMAIN_is_32bit_pv, struct domain, arch.is_32bit_pv); +#ifdef CONFIG_PV + OFFSET(DOMAIN_is_32bit_pv, struct domain, arch.pv.is_32bit); BLANK(); +#endif OFFSET(VCPUINFO_upcall_pending, struct vcpu_info, evtchn_upcall_pending); OFFSET(VCPUINFO_upcall_mask, struct vcpu_info, evtchn_upcall_mask); diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h index 4192c636b1..ae155d6522 100644 --- a/xen/include/asm-x86/domain.h +++ b/xen/include/asm-x86/domain.h @@ -254,6 +254,8 @@ struct pv_domain atomic_t nr_l4_pages; + /* Is a 32-bit PV guest? */ + bool is_32bit; /* XPTI active? */ bool xpti; /* Use PCID feature? */ @@ -333,8 +335,6 @@ struct arch_domain /* NB. protected by d->event_lock and by irq_desc[irq].lock */ struct radix_tree_root irq_pirq; - /* Is a 32-bit PV (non-HVM) guest? */ - bool_t is_32bit_pv; /* Is shared-info page in 32-bit format? */ bool_t has_32bit_shinfo; diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 195e7ee583..6101761d25 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -985,7 +985,11 @@ static always_inline bool is_pv_vcpu(const struct vcpu *v) #ifdef CONFIG_COMPAT static always_inline bool is_pv_32bit_domain(const struct domain *d) { - return is_pv_domain(d) && d->arch.is_32bit_pv; +#ifdef CONFIG_PV32 + return is_pv_domain(d) && d->arch.pv.is_32bit; +#else + return false; +#endif } static always_inline bool is_pv_32bit_vcpu(const struct vcpu *v) @@ -995,7 +999,14 @@ static always_inline bool is_pv_32bit_vcpu(const struct vcpu *v) static always_inline bool is_pv_64bit_domain(const struct domain *d) { - return is_pv_domain(d) && !d->arch.is_32bit_pv; + if ( !is_pv_domain(d) ) + return false; + +#ifdef CONFIG_PV32 + return !d->arch.pv.is_32bit; +#else + return true; +#endif } static always_inline bool is_pv_64bit_vcpu(const struct vcpu *v) From patchwork Fri Apr 17 15:50:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11495447 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BA75781 for ; Fri, 17 Apr 2020 15:51:16 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 966DF2087E for ; Fri, 17 Apr 2020 15:51:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="b6K8WYIT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 966DF2087E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jPTG8-0007XQ-VK; Fri, 17 Apr 2020 15:50:28 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jPTG7-0007X6-07 for xen-devel@lists.xenproject.org; Fri, 17 Apr 2020 15:50:27 +0000 X-Inumbo-ID: 1f61c946-80c3-11ea-8d2d-12813bfff9fa Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 1f61c946-80c3-11ea-8d2d-12813bfff9fa; Fri, 17 Apr 2020 15:50:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1587138613; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=I+Bzh3V08iUOv/zMH5/E2yogQfOh3a7ZmPE910Vqxec=; b=b6K8WYIT+yJsqi9Eb2Edl+tupQ8guogVzlVKRG4M/eAcnMCBmCVlo3kX vUlKY7W3ImYRftVfbgnHOnDwFmjEzm2CY0Ri/eeAGP8GQnpwgeI27+Aex gAK1p8aprKrYmAJsZPpZkETvmEXUegiXQpLusuRJt6enm/LJq72dFsvtM 4=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: ZZHdT/SVFo4ioFH9zsgNLB31zP9EFSLqKfJ1WrK6nWJd9gTlf5lGp94n5yIVrvpF+MJ6J6Z5xA quFbwGvXi+DAJP2wViJRg163Sf9NhLgrl02pS8kqn1Z4s8ixFd1VQxipQ80JL3rZyhr9FnHgFx RG2iB+SiTSaulJ8iyyT/KL/KlHIdN3fW+iyEmmrwhoPMa2pM+DDubLbuZmuFN1i3FF/9YE0li5 Mye8StftOunlmrc6BQr6EmnZKN53rV39LgrhQj3uIY/0aR+MeBj1NjRVTPyfHT17ZphWVikUnl NPw= X-SBRS: 2.7 X-MesageID: 16168815 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.72,395,1580792400"; d="scan'208";a="16168815" From: Andrew Cooper To: Xen-devel Subject: [PATCH 3/3] x86/pv: Compile out compat_gdt in !CONFIG_PV builds Date: Fri, 17 Apr 2020 16:50:04 +0100 Message-ID: <20200417155004.16806-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200417155004.16806-1-andrew.cooper3@citrix.com> References: <20200417155004.16806-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" There is no need for the Compat GDT if there are no 32bit PV guests. This saves 4k per online CPU Bloat-o-meter reports the following savings in Xen itself: add/remove: 0/3 grow/shrink: 1/4 up/down: 7/-4612 (-4605) Function old new delta cpu_smpboot_free 1249 1256 +7 per_cpu__compat_gdt_l1e 8 - -8 per_cpu__compat_gdt 8 - -8 init_idt_traps 442 420 -22 load_system_tables 414 364 -50 trap_init 444 280 -164 cpu_smpboot_callback 1255 991 -264 boot_compat_gdt 4096 - -4096 Total: Before=3062726, After=3058121, chg -0.15% Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné The increase in cpu_smpboot_free() appears to be a consequence of a totally different layout of basic blocks. --- xen/arch/x86/cpu/common.c | 5 +++-- xen/arch/x86/desc.c | 2 ++ xen/arch/x86/smpboot.c | 5 ++++- xen/arch/x86/traps.c | 10 +++++++--- 4 files changed, 16 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c index 1b33f1ed71..7b093cb421 100644 --- a/xen/arch/x86/cpu/common.c +++ b/xen/arch/x86/cpu/common.c @@ -752,8 +752,9 @@ void load_system_tables(void) _set_tssldt_desc(gdt + TSS_ENTRY, (unsigned long)tss, sizeof(*tss) - 1, SYS_DESC_tss_avail); - _set_tssldt_desc(compat_gdt + TSS_ENTRY, (unsigned long)tss, - sizeof(*tss) - 1, SYS_DESC_tss_busy); + if ( IS_ENABLED(CONFIG_PV32) ) + _set_tssldt_desc(compat_gdt + TSS_ENTRY, (unsigned long)tss, + sizeof(*tss) - 1, SYS_DESC_tss_busy); per_cpu(full_gdt_loaded, cpu) = false; lgdt(&gdtr); diff --git a/xen/arch/x86/desc.c b/xen/arch/x86/desc.c index dfeb1beaa8..39080ca672 100644 --- a/xen/arch/x86/desc.c +++ b/xen/arch/x86/desc.c @@ -55,6 +55,7 @@ seg_desc_t boot_gdt[PAGE_SIZE / sizeof(seg_desc_t)] = [SEL2GDT(PER_CPU_SELECTOR)] = { 0x0000910000000000 }, }; +#ifdef CONFIG_PV32 __section(".data.page_aligned") __aligned(PAGE_SIZE) seg_desc_t boot_compat_gdt[PAGE_SIZE / sizeof(seg_desc_t)] = { @@ -83,6 +84,7 @@ seg_desc_t boot_compat_gdt[PAGE_SIZE / sizeof(seg_desc_t)] = /* 0xe060 - per-CPU entry (limit == cpu) */ [SEL2GDT(PER_CPU_SELECTOR)] = { 0x0000910000000000 }, }; +#endif /* * Used by each CPU as it starts up, to enter C with a suitable %cs. diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index 09264b02d1..f9f63e496f 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -959,7 +959,8 @@ static void cpu_smpboot_free(unsigned int cpu, bool remove) free_domheap_page(mfn_to_page(mfn)); } - FREE_XENHEAP_PAGE(per_cpu(compat_gdt, cpu)); + if ( IS_ENABLED(CONFIG_PV32) ) + FREE_XENHEAP_PAGE(per_cpu(compat_gdt, cpu)); if ( remove ) { @@ -1001,6 +1002,7 @@ static int cpu_smpboot_alloc(unsigned int cpu) BUILD_BUG_ON(NR_CPUS > 0x10000); gdt[PER_CPU_GDT_ENTRY - FIRST_RESERVED_GDT_ENTRY].a = cpu; +#ifdef CONFIG_PV32 per_cpu(compat_gdt, cpu) = gdt = alloc_xenheap_pages(0, memflags); if ( gdt == NULL ) goto out; @@ -1008,6 +1010,7 @@ static int cpu_smpboot_alloc(unsigned int cpu) l1e_from_pfn(virt_to_mfn(gdt), __PAGE_HYPERVISOR_RW); memcpy(gdt, boot_compat_gdt, NR_RESERVED_GDT_PAGES * PAGE_SIZE); gdt[PER_CPU_GDT_ENTRY - FIRST_RESERVED_GDT_ENTRY].a = cpu; +#endif if ( idt_tables[cpu] == NULL ) idt_tables[cpu] = alloc_xenheap_pages(0, memflags); diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index e838846c6b..0bcf554e93 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -100,8 +100,10 @@ static DEFINE_PER_CPU(unsigned long, last_extable_addr); DEFINE_PER_CPU_READ_MOSTLY(seg_desc_t *, gdt); DEFINE_PER_CPU_READ_MOSTLY(l1_pgentry_t, gdt_l1e); +#ifdef CONFIG_PV32 DEFINE_PER_CPU_READ_MOSTLY(seg_desc_t *, compat_gdt); DEFINE_PER_CPU_READ_MOSTLY(l1_pgentry_t, compat_gdt_l1e); +#endif /* Master table, used by CPU0. */ idt_entry_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) @@ -1999,7 +2001,8 @@ void __init init_idt_traps(void) idt_tables[0] = idt_table; this_cpu(gdt) = boot_gdt; - this_cpu(compat_gdt) = boot_compat_gdt; + if ( IS_ENABLED(CONFIG_PV32) ) + this_cpu(compat_gdt) = boot_compat_gdt; } extern void (*const autogen_entrypoints[X86_NR_VECTORS])(void); @@ -2030,8 +2033,9 @@ void __init trap_init(void) /* Cache {,compat_}gdt_l1e now that physically relocation is done. */ this_cpu(gdt_l1e) = l1e_from_pfn(virt_to_mfn(boot_gdt), __PAGE_HYPERVISOR_RW); - this_cpu(compat_gdt_l1e) = - l1e_from_pfn(virt_to_mfn(boot_compat_gdt), __PAGE_HYPERVISOR_RW); + if ( IS_ENABLED(CONFIG_PV32) ) + this_cpu(compat_gdt_l1e) = + l1e_from_pfn(virt_to_mfn(boot_compat_gdt), __PAGE_HYPERVISOR_RW); percpu_traps_init();