From patchwork Mon Sep 30 13:32:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 11166731 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B4A541709 for ; Mon, 30 Sep 2019 13:34:49 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 853C5215EA for ; Mon, 30 Sep 2019 13:34:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="Wi8rBQLa" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 853C5215EA Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iEvnx-0005bU-W9; Mon, 30 Sep 2019 13:33:33 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iEvnx-0005ah-60 for xen-devel@lists.xenproject.org; Mon, 30 Sep 2019 13:33:33 +0000 X-Inumbo-ID: de9173d0-e386-11e9-97fb-bc764e2007e4 Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175]) by localhost (Halon) with ESMTPS id de9173d0-e386-11e9-97fb-bc764e2007e4; Mon, 30 Sep 2019 13:33:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1569850400; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sbKQ4jAblxD7x5cKYMbpA0wwvvCtJS1TSr1PWb+yeXc=; b=Wi8rBQLaJwLwGkfMZzddd1fjRA3oyizMbMFjGvDOVef4aZkumDajZh2r 2tb0akAaFt6xbyfHwjMBo3vqvKPvMZ384ee792mx49UIcy2BpcQ4YzOET CX0X3gDLnPKSaZCdIPz5rJAZa/14IYP9SuzeZivPnIj1X8bV7j/FPo6qY w=; Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa6.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ~all" Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: GZBQF3Qpmjs9kB/hvdGzhRHLcmlTXffaWozJaaWQLNCAFUpmU657/6FM4WIOoc0qfCXkJ1+ZFp mW5Y8ecEoLwxbGA1y7XFt4hCWvKrHBIAuFjj5lckhrMCnPCvX4mDjxr6YyfUmI95/XRQcV7ZR9 A59vJIIZPKeexdWiRCsGUK77ipqpZ64/48PUJddvaChbelSqT6TZvB8ellWZ01LBtjsdf7+8sA pinlbWeoc9Dgvz0Lrk6qpPNASVHsha26cxG5K6Lk1+a3QWXKnefVTbstvnIu5ajdM6NiWtq6Si mNU= X-SBRS: 2.7 X-MesageID: 6538595 X-Ironport-Server: esa6.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.64,567,1559534400"; d="scan'208";a="6538595" From: Roger Pau Monne To: Date: Mon, 30 Sep 2019 15:32:35 +0200 Message-ID: <20190930133238.49868-8-roger.pau@citrix.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190930133238.49868-1-roger.pau@citrix.com> References: <20190930133238.49868-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v3 07/10] ioreq: allow decoding accesses to MMCFG regions X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Paul Durrant , Wei Liu , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Pick up on the infrastructure already added for vPCI and allow ioreq to decode accesses to MMCFG regions registered for a domain. This infrastructure is still only accessible from internal callers, so MMCFG regions can only be registered from the internal domain builder used by PVH dom0. Note that the vPCI infrastructure to decode and handle accesses to MMCFG regions will be removed in following patches when vPCI is switched to become an internal ioreq server. Signed-off-by: Roger Pau Monné Reviewed-by: Paul Durrant --- Changes since v2: - Don't prevent mapping MCFG ranges by ioreq servers. Changes since v1: - Remove prototype for destroy_vpci_mmcfg. - Keep the code in io.c so PCI accesses to MMCFG regions can be decoded before ioreq processing. --- xen/arch/x86/hvm/dom0_build.c | 8 +-- xen/arch/x86/hvm/hvm.c | 2 +- xen/arch/x86/hvm/io.c | 79 ++++++++++++----------------- xen/arch/x86/hvm/ioreq.c | 18 +++++-- xen/arch/x86/physdev.c | 5 +- xen/drivers/passthrough/x86/iommu.c | 2 +- xen/include/asm-x86/hvm/io.h | 29 ++++++++--- 7 files changed, 75 insertions(+), 68 deletions(-) diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c index 831325150b..b30042d8f3 100644 --- a/xen/arch/x86/hvm/dom0_build.c +++ b/xen/arch/x86/hvm/dom0_build.c @@ -1108,10 +1108,10 @@ static void __hwdom_init pvh_setup_mmcfg(struct domain *d) for ( i = 0; i < pci_mmcfg_config_num; i++ ) { - rc = register_vpci_mmcfg_handler(d, pci_mmcfg_config[i].address, - pci_mmcfg_config[i].start_bus_number, - pci_mmcfg_config[i].end_bus_number, - pci_mmcfg_config[i].pci_segment); + rc = hvm_register_mmcfg(d, pci_mmcfg_config[i].address, + pci_mmcfg_config[i].start_bus_number, + pci_mmcfg_config[i].end_bus_number, + pci_mmcfg_config[i].pci_segment); if ( rc ) printk("Unable to setup MMCFG handler at %#lx for segment %u\n", pci_mmcfg_config[i].address, diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index c22cb39cf3..5348186c0c 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -753,7 +753,7 @@ void hvm_domain_destroy(struct domain *d) xfree(ioport); } - destroy_vpci_mmcfg(d); + hvm_free_mmcfg(d); } static int hvm_save_tsc_adjust(struct vcpu *v, hvm_domain_context_t *h) diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c index a5b0a23f06..3334888136 100644 --- a/xen/arch/x86/hvm/io.c +++ b/xen/arch/x86/hvm/io.c @@ -279,6 +279,18 @@ unsigned int hvm_pci_decode_addr(unsigned int cf8, unsigned int addr, return CF8_ADDR_LO(cf8) | (addr & 3); } +unsigned int hvm_mmcfg_decode_addr(const struct hvm_mmcfg *mmcfg, + paddr_t addr, pci_sbdf_t *sbdf) +{ + addr -= mmcfg->addr; + sbdf->bdf = MMCFG_BDF(addr); + sbdf->bus += mmcfg->start_bus; + sbdf->seg = mmcfg->segment; + + return addr & (PCI_CFG_SPACE_EXP_SIZE - 1); +} + + /* Do some sanity checks. */ static bool vpci_access_allowed(unsigned int reg, unsigned int len) { @@ -383,50 +395,14 @@ void register_vpci_portio_handler(struct domain *d) handler->ops = &vpci_portio_ops; } -struct hvm_mmcfg { - struct list_head next; - paddr_t addr; - unsigned int size; - uint16_t segment; - uint8_t start_bus; -}; - /* Handlers to trap PCI MMCFG config accesses. */ -static const struct hvm_mmcfg *vpci_mmcfg_find(const struct domain *d, - paddr_t addr) -{ - const struct hvm_mmcfg *mmcfg; - - list_for_each_entry ( mmcfg, &d->arch.hvm.mmcfg_regions, next ) - if ( addr >= mmcfg->addr && addr < mmcfg->addr + mmcfg->size ) - return mmcfg; - - return NULL; -} - -bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr) -{ - return vpci_mmcfg_find(d, addr); -} - -static unsigned int vpci_mmcfg_decode_addr(const struct hvm_mmcfg *mmcfg, - paddr_t addr, pci_sbdf_t *sbdf) -{ - addr -= mmcfg->addr; - sbdf->bdf = MMCFG_BDF(addr); - sbdf->bus += mmcfg->start_bus; - sbdf->seg = mmcfg->segment; - - return addr & (PCI_CFG_SPACE_EXP_SIZE - 1); -} - static int vpci_mmcfg_accept(struct vcpu *v, unsigned long addr) { struct domain *d = v->domain; bool found; read_lock(&d->arch.hvm.mmcfg_lock); - found = vpci_mmcfg_find(d, addr); + found = hvm_is_mmcfg_address(d, addr); read_unlock(&d->arch.hvm.mmcfg_lock); return found; @@ -443,14 +419,14 @@ static int vpci_mmcfg_read(struct vcpu *v, unsigned long addr, *data = ~0ul; read_lock(&d->arch.hvm.mmcfg_lock); - mmcfg = vpci_mmcfg_find(d, addr); + mmcfg = hvm_mmcfg_find(d, addr); if ( !mmcfg ) { read_unlock(&d->arch.hvm.mmcfg_lock); return X86EMUL_RETRY; } - reg = vpci_mmcfg_decode_addr(mmcfg, addr, &sbdf); + reg = hvm_mmcfg_decode_addr(mmcfg, addr, &sbdf); read_unlock(&d->arch.hvm.mmcfg_lock); if ( !vpci_access_allowed(reg, len) || @@ -485,14 +461,14 @@ static int vpci_mmcfg_write(struct vcpu *v, unsigned long addr, pci_sbdf_t sbdf; read_lock(&d->arch.hvm.mmcfg_lock); - mmcfg = vpci_mmcfg_find(d, addr); + mmcfg = hvm_mmcfg_find(d, addr); if ( !mmcfg ) { read_unlock(&d->arch.hvm.mmcfg_lock); return X86EMUL_RETRY; } - reg = vpci_mmcfg_decode_addr(mmcfg, addr, &sbdf); + reg = hvm_mmcfg_decode_addr(mmcfg, addr, &sbdf); read_unlock(&d->arch.hvm.mmcfg_lock); if ( !vpci_access_allowed(reg, len) || @@ -512,9 +488,9 @@ static const struct hvm_mmio_ops vpci_mmcfg_ops = { .write = vpci_mmcfg_write, }; -int register_vpci_mmcfg_handler(struct domain *d, paddr_t addr, - unsigned int start_bus, unsigned int end_bus, - unsigned int seg) +int hvm_register_mmcfg(struct domain *d, paddr_t addr, + unsigned int start_bus, unsigned int end_bus, + unsigned int seg) { struct hvm_mmcfg *mmcfg, *new; @@ -549,7 +525,7 @@ int register_vpci_mmcfg_handler(struct domain *d, paddr_t addr, return ret; } - if ( list_empty(&d->arch.hvm.mmcfg_regions) ) + if ( list_empty(&d->arch.hvm.mmcfg_regions) && has_vpci(d) ) register_mmio_handler(d, &vpci_mmcfg_ops); list_add(&new->next, &d->arch.hvm.mmcfg_regions); @@ -558,7 +534,7 @@ int register_vpci_mmcfg_handler(struct domain *d, paddr_t addr, return 0; } -void destroy_vpci_mmcfg(struct domain *d) +void hvm_free_mmcfg(struct domain *d) { struct list_head *mmcfg_regions = &d->arch.hvm.mmcfg_regions; @@ -574,6 +550,17 @@ void destroy_vpci_mmcfg(struct domain *d) write_unlock(&d->arch.hvm.mmcfg_lock); } +const struct hvm_mmcfg *hvm_mmcfg_find(const struct domain *d, paddr_t addr) +{ + const struct hvm_mmcfg *mmcfg; + + list_for_each_entry ( mmcfg, &d->arch.hvm.mmcfg_regions, next ) + if ( addr >= mmcfg->addr && addr < mmcfg->addr + mmcfg->size ) + return mmcfg; + + return NULL; +} + /* * Local variables: * mode: C diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c index 57719c607c..6b87a55db5 100644 --- a/xen/arch/x86/hvm/ioreq.c +++ b/xen/arch/x86/hvm/ioreq.c @@ -1326,27 +1326,34 @@ ioservid_t hvm_select_ioreq_server(struct domain *d, ioreq_t *p) uint8_t type; uint64_t addr; unsigned int id; + const struct hvm_mmcfg *mmcfg; if ( p->type != IOREQ_TYPE_COPY && p->type != IOREQ_TYPE_PIO ) return XEN_INVALID_IOSERVID; cf8 = d->arch.hvm.pci_cf8; - if ( p->type == IOREQ_TYPE_PIO && - (p->addr & ~3) == 0xcfc && - CF8_ENABLED(cf8) ) + read_lock(&d->arch.hvm.mmcfg_lock); + if ( (p->type == IOREQ_TYPE_PIO && + (p->addr & ~3) == 0xcfc && + CF8_ENABLED(cf8)) || + (p->type == IOREQ_TYPE_COPY && + (mmcfg = hvm_mmcfg_find(d, p->addr)) != NULL) ) { uint32_t x86_fam; pci_sbdf_t sbdf; unsigned int reg; - reg = hvm_pci_decode_addr(cf8, p->addr, &sbdf); + reg = p->type == IOREQ_TYPE_PIO ? hvm_pci_decode_addr(cf8, p->addr, + &sbdf) + : hvm_mmcfg_decode_addr(mmcfg, p->addr, + &sbdf); /* PCI config data cycle */ type = XEN_DMOP_IO_RANGE_PCI; addr = ((uint64_t)sbdf.sbdf << 32) | reg; /* AMD extended configuration space access? */ - if ( CF8_ADDR_HI(cf8) && + if ( p->type == IOREQ_TYPE_PIO && CF8_ADDR_HI(cf8) && d->arch.cpuid->x86_vendor == X86_VENDOR_AMD && (x86_fam = get_cpu_family( d->arch.cpuid->basic.raw_fms, NULL, NULL)) > 0x10 && @@ -1365,6 +1372,7 @@ ioservid_t hvm_select_ioreq_server(struct domain *d, ioreq_t *p) XEN_DMOP_IO_RANGE_PORT : XEN_DMOP_IO_RANGE_MEMORY; addr = p->addr; } + read_unlock(&d->arch.hvm.mmcfg_lock); FOR_EACH_IOREQ_SERVER(d, id, s) { diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c index 3a3c15890b..f61f66df5f 100644 --- a/xen/arch/x86/physdev.c +++ b/xen/arch/x86/physdev.c @@ -562,9 +562,8 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) * For HVM (PVH) domains try to add the newly found MMCFG to the * domain. */ - ret = register_vpci_mmcfg_handler(currd, info.address, - info.start_bus, info.end_bus, - info.segment); + ret = hvm_register_mmcfg(currd, info.address, info.start_bus, + info.end_bus, info.segment); } break; diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c index 59905629e1..53cdbb45f0 100644 --- a/xen/drivers/passthrough/x86/iommu.c +++ b/xen/drivers/passthrough/x86/iommu.c @@ -152,7 +152,7 @@ static bool __hwdom_init hwdom_iommu_map(const struct domain *d, * TODO: runtime added MMCFG regions are not checked to make sure they * don't overlap with already mapped regions, thus preventing trapping. */ - if ( has_vpci(d) && vpci_is_mmcfg_address(d, pfn_to_paddr(pfn)) ) + if ( has_vpci(d) && hvm_is_mmcfg_address(d, pfn_to_paddr(pfn)) ) return false; return true; diff --git a/xen/include/asm-x86/hvm/io.h b/xen/include/asm-x86/hvm/io.h index 7ceb119b64..86ebbd1e7e 100644 --- a/xen/include/asm-x86/hvm/io.h +++ b/xen/include/asm-x86/hvm/io.h @@ -165,9 +165,19 @@ void stdvga_deinit(struct domain *d); extern void hvm_dpci_msi_eoi(struct domain *d, int vector); -/* Decode a PCI port IO access into a bus/slot/func/reg. */ +struct hvm_mmcfg { + struct list_head next; + paddr_t addr; + unsigned int size; + uint16_t segment; + uint8_t start_bus; +}; + +/* Decode a PCI port IO or MMCFG access into a bus/slot/func/reg. */ unsigned int hvm_pci_decode_addr(unsigned int cf8, unsigned int addr, pci_sbdf_t *sbdf); +unsigned int hvm_mmcfg_decode_addr(const struct hvm_mmcfg *mmcfg, + paddr_t addr, pci_sbdf_t *sbdf); /* * HVM port IO handler that performs forwarding of guest IO ports into machine @@ -178,15 +188,18 @@ void register_g2m_portio_handler(struct domain *d); /* HVM port IO handler for vPCI accesses. */ void register_vpci_portio_handler(struct domain *d); -/* HVM MMIO handler for PCI MMCFG accesses. */ -int register_vpci_mmcfg_handler(struct domain *d, paddr_t addr, - unsigned int start_bus, unsigned int end_bus, - unsigned int seg); -/* Destroy tracked MMCFG areas. */ -void destroy_vpci_mmcfg(struct domain *d); +/* HVM PCI MMCFG regions registration. */ +int hvm_register_mmcfg(struct domain *d, paddr_t addr, + unsigned int start_bus, unsigned int end_bus, + unsigned int seg); +void hvm_free_mmcfg(struct domain *d); +const struct hvm_mmcfg *hvm_mmcfg_find(const struct domain *d, paddr_t addr); /* Check if an address is between a MMCFG region for a domain. */ -bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr); +static inline bool hvm_is_mmcfg_address(const struct domain *d, paddr_t addr) +{ + return hvm_mmcfg_find(d, addr); +} #endif /* __ASM_X86_HVM_IO_H__ */