From patchwork Thu Nov 21 15:54:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 11256455 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C6EA26C1 for ; Thu, 21 Nov 2019 15:55:06 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A28B020692 for ; Thu, 21 Nov 2019 15:55:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="TJCdrUtl" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A28B020692 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iXomp-0007No-UP; Thu, 21 Nov 2019 15:54:27 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iXomo-0007NH-8h for xen-devel@lists.xenproject.org; Thu, 21 Nov 2019 15:54:26 +0000 X-Inumbo-ID: 2ea9cb9c-0c77-11ea-9631-bc764e2007e4 Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 2ea9cb9c-0c77-11ea-9631-bc764e2007e4; Thu, 21 Nov 2019 15:54:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1574351661; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JDwT4M3xxpK5idmTqO4NNf3501J90gXqRN6QCmLvuR4=; b=TJCdrUtlJriTGZT/Sr4vgqZiMlr7+lotC8P4cDqs8ztYk6hc55aFloMk qAFMxkfeVs9gINvvaijnjAMoelNB0LKYSo2bYFiym4FphqbhBoPCiw3hz r57HfHpQ+JwAbSvx6cs98I0F39uKB0tdjIu5o96GvsK5BEAwtOmvVb8i7 Q=; Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa4.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa4.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa4.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: R/2yAziJ4wFrsoOyaDMU9g6IefXn3HPNFcBsl9C6AtxSsrPYhbc1XV7clN2WwCHcqDveTMJJvs OZxK4qTRtSgZc/eEf5ZNsBarb8miF+pOdkEWuyOfDdN6Bn/4ISlItX3lITkB0/zjVae7f2MqVN GLCRQkrdkKYqeVN/PvQ8rKxpEh8msBwxAKPI1O/nYPXh8KlTw626nRchhmaB7gAYVXrluVUSti XLg0TO4+j1dfeqlV/Q3bSvtw5/XedCXOn++OBlz7QQXJuSp6Gq5ekntBCSJ4cZxPx5AvWaVLJM 3Kg= X-SBRS: 2.7 X-MesageID: 9206662 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,226,1571716800"; d="scan'208";a="9206662" From: Roger Pau Monne To: Date: Thu, 21 Nov 2019 16:54:09 +0100 Message-ID: <20191121155411.13250-2-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191121155411.13250-1-roger.pau@citrix.com> References: <20191121155411.13250-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH 1/3] x86/ioapic: only use dest32 with x2apic and interrupt remapping enabled X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The IO-APIC code assumes that x2apic being enabled also implies interrupt remapping being enabled, and hence will use the 32bit destination field in the IO-APIC entry. This is safe now, but there's no reason to not enable x2APIC even without interrupt remapping, and hence the IO-APIC code needs to use the 32 bit destination field only when both interrupt remapping and x2APIC are enabled. Signed-off-by: Roger Pau Monné --- xen/arch/x86/io_apic.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c index 732b57995c..9d38568fba 100644 --- a/xen/arch/x86/io_apic.c +++ b/xen/arch/x86/io_apic.c @@ -964,7 +964,7 @@ static hw_irq_controller ioapic_edge_type; #define IOAPIC_LEVEL 1 #define SET_DEST(ent, mode, val) do { \ - if (x2apic_enabled) \ + if (x2apic_enabled && iommu_intremap) \ (ent).dest.dest32 = (val); \ else \ (ent).dest.mode.mode##_dest = (val); \ @@ -1194,14 +1194,14 @@ static void /*__init*/ __print_IO_APIC(bool boot) printk(KERN_DEBUG ".... IRQ redirection table:\n"); printk(KERN_DEBUG " NR %s Msk Trg IRR Pol Stat DstM DelM Vec\n", - x2apic_enabled ? " DestID" : "Dst"); + (x2apic_enabled && iommu_intremap) ? " DestID" : "Dst"); for (i = 0; i <= reg_01.bits.entries; i++) { struct IO_APIC_route_entry entry; entry = ioapic_read_entry(apic, i, 0); - if ( x2apic_enabled ) + if ( x2apic_enabled && iommu_intremap ) printk(KERN_DEBUG " %02x %08x", i, entry.dest.dest32); else printk(KERN_DEBUG " %02x %02x ", i, @@ -2504,9 +2504,9 @@ void dump_ioapic_irq_info(void) rte.dest_mode ? 'L' : 'P', rte.delivery_status, rte.polarity, rte.irr, rte.trigger ? 'L' : 'E', rte.mask, - x2apic_enabled ? 8 : 2, - x2apic_enabled ? rte.dest.dest32 - : rte.dest.logical.logical_dest); + (x2apic_enabled && iommu_intremap) ? 8 : 2, + (x2apic_enabled && iommu_intremap) ? + rte.dest.dest32 : rte.dest.logical.logical_dest); if ( entry->next == 0 ) break; From patchwork Thu Nov 21 15:54:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 11256453 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2C2CF6C1 for ; Thu, 21 Nov 2019 15:55:05 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 077E920692 for ; Thu, 21 Nov 2019 15:55:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="hkzZgbdG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 077E920692 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iXomn-0007Mq-Lz; Thu, 21 Nov 2019 15:54:25 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iXomm-0007Md-HM for xen-devel@lists.xenproject.org; Thu, 21 Nov 2019 15:54:24 +0000 X-Inumbo-ID: 2fecc9aa-0c77-11ea-a33c-12813bfff9fa Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 2fecc9aa-0c77-11ea-a33c-12813bfff9fa; Thu, 21 Nov 2019 15:54:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1574351663; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ni72YK4MQ3tW3zR5VLAk3DqhGd0LLmGdxnb57VoU8vc=; b=hkzZgbdGlz4ihJDLOQ1L5uxu/TElH+5So3rgGxhrDKGT6Md1z1vVNJHT ImNWNtP6R/I6wtRbPVfV2OwMzgB40tLmDYMwOHTEGIIDMDPZFN3cQrADO FxqozeThHN8CMfzd/dGy/m8nQEwaVMuL6odc7cZSBXFF8oRIfFfUcgd9B U=; Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa2.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: 4zu0VCs2s2QXTatdlqnu7bLiRuguGvbbG13Gmc3um6HIFl2QAtukvTYP8RLL9f5XFDzIOK2PqF vdRVC7bR3+IBHaFnZp/sB1tMuKE2/LFdn2bGuA50FjLowRPDmI0AEtrZLDO+RkblNqRlUXuAco 1hBh1BHnx6HHCoibsp/dnB+Q6e+nuASRqFQhwARyHSBoF5fbIFo7w3qzi7UnhBz7BqJDWxj0uh r5mf/88y/Mw6fzkswWXdibL/D4gao+cH94/tpMjt9zTnRtbZe+nJ4HkM0eIjZi5yTJ3PnQtZi/ cXI= X-SBRS: 2.7 X-MesageID: 8668388 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,226,1571716800"; d="scan'208";a="8668388" From: Roger Pau Monne To: Date: Thu, 21 Nov 2019 16:54:10 +0100 Message-ID: <20191121155411.13250-3-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191121155411.13250-1-roger.pau@citrix.com> References: <20191121155411.13250-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH 2/3] x86/smp: check APIC ID on AP bringup X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Check that the processor to be woken up APIC ID is addressable in the current APIC mode. Note that in practice systems with APIC IDs > 255 should already have x2APIC enabled by the firmware, and hence this is mostly a safety belt. Signed-off-by: Roger Pau Monné --- xen/arch/x86/smpboot.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index f86c15bde3..4e9fe7e03e 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -1315,6 +1315,13 @@ int __cpu_up(unsigned int cpu) if ( (apicid = x86_cpu_to_apicid[cpu]) == BAD_APICID ) return -ENODEV; + if ( (!x2apic_enabled || !iommu_intremap) && (apicid >> 8) ) + { + printk("Processor with APIC ID %u cannot be onlined in xAPIC mode " + "or without interrupt remapping\n", apicid); + return -EINVAL; + } + if ( (ret = do_boot_cpu(apicid, cpu)) != 0 ) return ret; From patchwork Thu Nov 21 15:54:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 11256457 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1EE416C1 for ; Thu, 21 Nov 2019 15:55:13 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EEC0320692 for ; Thu, 21 Nov 2019 15:55:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="UdBNRAq/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EEC0320692 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iXoms-0007P3-7g; Thu, 21 Nov 2019 15:54:30 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iXomr-0007Ok-HL for xen-devel@lists.xenproject.org; Thu, 21 Nov 2019 15:54:29 +0000 X-Inumbo-ID: 30f6b703-0c77-11ea-a33c-12813bfff9fa Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 30f6b703-0c77-11ea-a33c-12813bfff9fa; Thu, 21 Nov 2019 15:54:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1574351666; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dJyfm/pH7j3NdbhfezdHiEx15uqqnCPzcffRcjBjEhc=; b=UdBNRAq/+59S+l7DidNOaSn/9FhabQmGS3Rce/ZZxs4V84s9iziB27O2 OAujtj8fUNxphfApXjaQWe/QudJ0tZi3cKUXmXHCmtyXWvyA3tBy5URlU e4+AtdA1GWPud/lIKpWCUE0mzUTkCnxYb23RdoVJ0cf9JgRmWGaUSAA32 U=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: xXOj58UCErW4JR+0l8vJJoaJIC0w+A1Jp3FvZYaj+xws/gMxQ/E4lNUqB5wsYuUDeM5LVpQBLy kT2g75cAi8uqT+tk6sqe1+tIQ4GT8WKZ8aVhxdh+zXdC3fOuCb9NIm6dteF5wegO/NpuH+wyJ7 kRW80vHy9ielT9o/7u4eKQpAoPzImnCkyo2GKEyMRdrP4fdXNo528/KDYtVvdrRUzAJJWB+6Mu bojl40hz3pI9mpNG63S3YW3OKi2pwjugOx7eY9TqKvKNJLxpJNsR7yEuZosEaqR3yuV3jPzPdG FaM= X-SBRS: 2.7 X-MesageID: 9021137 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,226,1571716800"; d="scan'208";a="9021137" From: Roger Pau Monne To: Date: Thu, 21 Nov 2019 16:54:11 +0100 Message-ID: <20191121155411.13250-4-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191121155411.13250-1-roger.pau@citrix.com> References: <20191121155411.13250-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH 3/3] x86/apic: allow enabling x2APIC mode regardless of interrupt remapping X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" x2APIC mode doesn't mandate interrupt remapping, and hence can be enabled independently. This patch enables x2APIC when available, regardless of whether there's interrupt remapping support. This is beneficial specially when running on virtualized environments, since it reduces the amount of vmexits. For example when sending an IPI in xAPIC mode Xen performs at least 3 different accesses to the APIC MMIO region, while when using x2APIC mode a single wrmsr is used. The following numbers are from a lock profiling of a Xen PV shim running a Linux PV kernel with 32 vCPUs and xAPIC mode: (XEN) Global lock flush_lock: addr=ffff82d0804af1c0, lockval=03190319, not locked (XEN) lock:656153(892606463454), block:602183(9495067321843) Average lock time: 1360363ns Average block time: 15767743ns While the following are from the same configuration but with the shim using x2APIC mode: (XEN) Global lock flush_lock: addr=ffff82d0804b01c0, lockval=1adb1adb, not locked (XEN) lock:1841883(1375128998543), block:1658716(10193054890781) Average lock time: 746588ns Average block time: 6145147ns Enabling x2APIC has halved the average lock time, thus reducing contention. Signed-off-by: Roger Pau Monné --- xen/arch/x86/apic.c | 94 ++++++++++++++++++++++----------------------- 1 file changed, 46 insertions(+), 48 deletions(-) diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c index a5f7b05d5a..bac682396f 100644 --- a/xen/arch/x86/apic.c +++ b/xen/arch/x86/apic.c @@ -492,7 +492,8 @@ static void __enable_x2apic(void) static void resume_x2apic(void) { - iommu_enable_x2apic(); + if ( iommu_supports_x2apic() ) + iommu_enable_x2apic(); __enable_x2apic(); } @@ -695,7 +696,8 @@ int lapic_suspend(void) local_irq_save(flags); disable_local_APIC(); - iommu_disable_x2apic(); + if ( iommu_supports_x2apic() ) + iommu_disable_x2apic(); local_irq_restore(flags); return 0; } @@ -875,56 +877,49 @@ void __init x2apic_bsp_setup(void) printk("x2APIC: Already enabled by BIOS: Ignoring cmdline disable.\n"); } - if ( !iommu_supports_x2apic() ) + if ( iommu_supports_x2apic() ) { - if ( !x2apic_enabled ) + if ( (ioapic_entries = alloc_ioapic_entries()) == NULL ) { - printk("Not enabling x2APIC: depends on IOMMU support\n"); - return; + printk("Allocate ioapic_entries failed\n"); + goto out; } - panic("x2APIC: already enabled by BIOS, but no IOMMU support\n"); - } - - if ( (ioapic_entries = alloc_ioapic_entries()) == NULL ) - { - printk("Allocate ioapic_entries failed\n"); - goto out; - } - if ( save_IO_APIC_setup(ioapic_entries) ) - { - printk("Saving IO-APIC state failed\n"); - goto out; - } + if ( save_IO_APIC_setup(ioapic_entries) ) + { + printk("Saving IO-APIC state failed\n"); + goto out; + } - mask_8259A(); - mask_IO_APIC_setup(ioapic_entries); + mask_8259A(); + mask_IO_APIC_setup(ioapic_entries); - switch ( iommu_enable_x2apic() ) - { - case 0: - intremap_enabled = true; - break; - case -ENXIO: /* ACPI_DMAR_X2APIC_OPT_OUT set */ - if ( !x2apic_enabled ) + switch ( iommu_enable_x2apic() ) { - printk("Not enabling x2APIC (upon firmware request)\n"); + case 0: + intremap_enabled = true; + break; + case -ENXIO: /* ACPI_DMAR_X2APIC_OPT_OUT set */ + if ( !x2apic_enabled ) + { + printk("Not enabling x2APIC (upon firmware request)\n"); + intremap_enabled = false; + goto restore_out; + } + /* fall through */ + default: + if ( x2apic_enabled ) + panic("Interrupt remapping could not be enabled while " + "x2APIC is already enabled by BIOS\n"); + + printk(XENLOG_ERR + "Failed to enable Interrupt Remapping: Will not enable x2APIC.\n"); intremap_enabled = false; goto restore_out; } - /* fall through */ - default: - if ( x2apic_enabled ) - panic("Interrupt remapping could not be enabled while " - "x2APIC is already enabled by BIOS\n"); - - printk(XENLOG_ERR - "Failed to enable Interrupt Remapping: Will not enable x2APIC.\n"); - intremap_enabled = false; - goto restore_out; - } - force_iommu = 1; + force_iommu = 1; + } if ( !x2apic_enabled ) { @@ -938,13 +933,16 @@ void __init x2apic_bsp_setup(void) printk("Switched to APIC driver %s\n", genapic.name); restore_out: - /* - * NB: do not use raw mode when restoring entries if the iommu has been - * enabled during the process, because the entries need to be translated - * and added to the remapping table in that case. - */ - restore_IO_APIC_setup(ioapic_entries, !intremap_enabled); - unmask_8259A(); + if ( iommu_supports_x2apic() ) + { + /* + * NB: do not use raw mode when restoring entries if the iommu has been + * enabled during the process, because the entries need to be translated + * and added to the remapping table in that case. + */ + restore_IO_APIC_setup(ioapic_entries, !intremap_enabled); + unmask_8259A(); + } out: if ( ioapic_entries )