From patchwork Wed Aug 28 15:24:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Igor Druzhinin X-Patchwork-Id: 11119343 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7E176112C for ; Wed, 28 Aug 2019 15:25:55 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5A3102077B for ; Wed, 28 Aug 2019 15:25:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="b32c/w5H" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5A3102077B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i2zoD-0003qS-JM; Wed, 28 Aug 2019 15:24:29 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i2zoC-0003qM-0F for xen-devel@lists.xenproject.org; Wed, 28 Aug 2019 15:24:28 +0000 X-Inumbo-ID: ebdce482-c9a7-11e9-8980-bc764e2007e4 Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id ebdce482-c9a7-11e9-8980-bc764e2007e4; Wed, 28 Aug 2019 15:24:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1567005868; h=from:to:cc:subject:date:message-id:mime-version; bh=lc2lizEH0i8KXPIv7qbrd72Lzv3+tYP3EW/QpLEEIMU=; b=b32c/w5HkIuepKC+zRZ/aMIcwGXrBF8Ki7RP7fcSvKZIjuM6Tj/IFdNV H3Ho6r5e529WkVU59DjHbCY4/wOM+iambnDS6FZevV5IOKmfUH0H28pd3 Hs4OCqZMN3NYVr3fL2LmQmT35F8Y+SQstG+kS7aloCpSqzfHXYy4xUl2s M=; Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=igor.druzhinin@citrix.com; spf=Pass smtp.mailfrom=igor.druzhinin@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of igor.druzhinin@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="igor.druzhinin@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa2.hc3370-68.iphmx.com: domain of igor.druzhinin@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="igor.druzhinin@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ~all" Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: MM6OeNsdDOscL6xrvMDsHKfZqVLLJG+y7i5h+bCZ/EfyT4HAbgZMGfr5QhgxI/ijUKyz7CNAeG MVnLVUDQjvHbs6lUHgbCmmFgCSdBMQLz8UGdmYyvGJQoXQNYU7MJMkfc9hpmKNrTo4FHFkDgrk BSJUNSMVD+9EdZEe9NXUEV/x19j+lu5XQmH0WmYyG4gLt+RFhDriKeu63KDEn5Lm00vi8zkHX3 LV4vvgxdcPHc9Qe+ozGUk4aQIg520Wb7nhya96BRXx+YjbmpmPXRyf/d9zfT05NweVqcyJLWlU OKI= X-SBRS: 2.7 X-MesageID: 4827100 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.64,441,1559534400"; d="scan'208";a="4827100" From: Igor Druzhinin To: Date: Wed, 28 Aug 2019 16:24:22 +0100 Message-ID: <1567005862-18540-1-git-send-email-igor.druzhinin@citrix.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 Subject: [Xen-devel] [PATCH] x86/mmcfg: add "force" option for MCFG X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Igor Druzhinin , sstabellini@kernel.org, wl@xen.org, konrad.wilk@oracle.com, George.Dunlap@eu.citrix.com, andrew.cooper3@citrix.com, ian.jackson@eu.citrix.com, tim@xen.org, julien.grall@arm.com, jbeulich@suse.com, roger.pau@citrix.com Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" If MCFG area is not reserved in E820 Xen by default will defer its usage until Dom0 registers it explicitly after ACPI parser recognizes it as a reserved resource in DSDT. Having it reserved in E820 is not mandatory according to "PCI Firmware Specification, rev 3.2" (par. 4.1.2) and firmware is free to keep a hole E820 in that place. Unfortunately, keeping it disabled at this point makes Xen fail to recognize many of PCIe extended capabilities early enough for newly added devices. Namely, (1) some of VT-d quirks are not applied during Dom0 device handoff, (2) currently MCFG reservation report comes too late from Dom0 after some of PCI devices being registered in Xen by Dom0 kernel that break initialization of a number of PCIe capabilities (e.g. SR-IOV VF BAR sizing). Since introduction of ACPI parser in Xen is not possible add a "force" option that will allow Xen to use MCFG area even it's not properly reserved in E820. Signed-off-by: Igor Druzhinin --- I think we need to have this option to at least have a way to workaround problem (1). Problem (2) could be solved in Dom0 kernel by invoking xen_mcfg_late() earlier but before the first PCI bus enumertaion which currently happens somwhere during ACPI scan I think. The question is what the defult value for this option should be? --- docs/misc/xen-command-line.pandoc | 8 +++++--- xen/arch/x86/e820.c | 20 ++++++++++++++++++++ xen/arch/x86/x86_64/mmconfig-shared.c | 34 ++++++++++++++++++++-------------- xen/include/asm-x86/e820.h | 1 + 4 files changed, 46 insertions(+), 17 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index 7c72e31..f13b10c 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -1424,11 +1424,13 @@ ordinary DomU, control domain, hardware domain, and - when supported by the platform - DomU with pass-through device assigned). ### mmcfg (x86) -> `= [,amd-fam10]` +> `= List of [ , force, amd-fam10 ]` -> Default: `1` +> Default: `true,force` -Specify if the MMConfig space should be enabled. +Specify if the MMConfig space should be enabled. If force option is specified +(default) MMConfig space will be used by Xen early in boot even if it's +not reserved by firmware in the E820 table. ### mmio-relax (x86) > `= | all` diff --git a/xen/arch/x86/e820.c b/xen/arch/x86/e820.c index 8e8a2c4..edef72a 100644 --- a/xen/arch/x86/e820.c +++ b/xen/arch/x86/e820.c @@ -37,6 +37,26 @@ struct e820map e820; struct e820map __initdata e820_raw; /* + * This function checks if any part of the range is mapped + * with type. + */ +int __init e820_any_mapped(u64 start, u64 end, unsigned type) +{ + int i; + + for (i = 0; i < e820.nr_map; i++) { + struct e820entry *ei = &e820.map[i]; + + if (type && ei->type != type) + continue; + if (ei->addr >= end || ei->addr + ei->size <= start) + continue; + return 1; + } + return 0; +} + +/* * This function checks if the entire range is mapped with type. * * Note: this function only works correct if the e820 table is sorted and diff --git a/xen/arch/x86/x86_64/mmconfig-shared.c b/xen/arch/x86/x86_64/mmconfig-shared.c index cc08b52..9fc0865 100644 --- a/xen/arch/x86/x86_64/mmconfig-shared.c +++ b/xen/arch/x86/x86_64/mmconfig-shared.c @@ -26,33 +26,34 @@ #include "mmconfig.h" +static bool_t __read_mostly force_mmcfg = true; unsigned int pci_probe = PCI_PROBE_CONF1 | PCI_PROBE_MMCONF; static int __init parse_mmcfg(const char *s) { const char *ss; - int rc = 0; + int val, rc = 0; do { ss = strchr(s, ','); if ( !ss ) ss = strchr(s, '\0'); - switch ( parse_bool(s, ss) ) - { - case 0: - pci_probe &= ~PCI_PROBE_MMCONF; - break; - case 1: - break; - default: - if ( !cmdline_strcmp(s, "amd_fam10") || - !cmdline_strcmp(s, "amd-fam10") ) + if ( (val = parse_bool(s, ss)) >= 0 ) { + if ( val ) + pci_probe |= PCI_PROBE_MMCONF; + else + pci_probe &= ~PCI_PROBE_MMCONF; + } else if ( (val = parse_boolean("amd_fam10", s, ss)) >= 0 || + (val = parse_boolean("amd-fam10", s, ss)) >= 0 ) { + if ( val ) pci_probe |= PCI_CHECK_ENABLE_AMD_MMCONF; else - rc = -EINVAL; - break; - } + pci_probe &= ~PCI_CHECK_ENABLE_AMD_MMCONF; + } else if ( (val = parse_boolean("force", s, ss)) >= 0) + force_mmcfg = val; + else + rc = -EINVAL; s = ss + 1; } while ( *ss ); @@ -355,6 +356,11 @@ static int __init is_mmconf_reserved( (unsigned int)cfg->start_bus_number, (unsigned int)cfg->end_bus_number); } + } else if (!e820_any_mapped(addr, addr + old_size - 1, 0)) { + if (!force_mmcfg) + printk(KERN_WARNING "PCI: MCFG area at %lx is not reserved in E820, " + "use mmcfg=force option\n", addr); + valid = force_mmcfg; } return valid; diff --git a/xen/include/asm-x86/e820.h b/xen/include/asm-x86/e820.h index 52916fb..8babb4b 100644 --- a/xen/include/asm-x86/e820.h +++ b/xen/include/asm-x86/e820.h @@ -24,6 +24,7 @@ struct e820map { }; extern int sanitize_e820_map(struct e820entry *biosmap, unsigned int *pnr_map); +extern int e820_any_mapped(u64 start, u64 end, unsigned type); extern int e820_all_mapped(u64 start, u64 end, unsigned type); extern int reserve_e820_ram(struct e820map *e820, uint64_t s, uint64_t e); extern int e820_change_range_type(