From patchwork Tue Apr 11 10:04:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Roger Pau Monne X-Patchwork-Id: 9674849 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id DF59F60381 for ; Tue, 11 Apr 2017 10:08:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D5956284EF for ; Tue, 11 Apr 2017 10:08:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CA85528537; Tue, 11 Apr 2017 10:08:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EC2D428500 for ; Tue, 11 Apr 2017 10:08:11 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cxsgT-0006L0-9q; Tue, 11 Apr 2017 10:06:01 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cxsgR-0006JM-Ua for xen-devel@lists.xenproject.org; Tue, 11 Apr 2017 10:06:00 +0000 Received: from [85.158.143.35] by server-8.bemta-6.messagelabs.com id 0E/8E-03648-78AACE85; Tue, 11 Apr 2017 10:05:59 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrJIsWRWlGSWpSXmKPExsXitHRDpG7bqjc RBm+eGlh83zKZyYHR4/CHKywBjFGsmXlJ+RUJrBkX5kxmLFjvW/H82U6mBsb15l2MnBwSAv4S mzvvMIHYbAI6Ehfn7mTrYuTgEBFQkbi916CLkYuDWaCdWaJz+wFGkBphAReJx1da2EBsFgFVi UXnboD18gpYSvxteswEMVNP4u3EF2D1nAJWEvNOvQOrFwKq2XZiO1S9oMTJmU9YQGxmAU2J1u 2/2SFseYnmrbOZIeoVJfrnPWCbwMg3C0nLLCQts5C0LGBkXsWoUZxaVJZapGtoqJdUlJmeUZK bmJmja2hgppebWlycmJ6ak5hUrJecn7uJERhsDECwg/HTsoBDjJIcTEqivAEzX0cI8SXlp1Rm JBZnxBeV5qQWH2KU4eBQkuCds/JNhJBgUWp6akVaZg4w7GHSEhw8SiK8PiBp3uKCxNzizHSI1 ClGXY4rrR/fMwmx5OXnpUqJ8zKBFAmAFGWU5sGNgMXgJUZZKWFeRqCjhHgKUotyM0tQ5V8xin MwKgnz2oBM4cnMK4Hb9AroCCagI87seglyREkiQkqqgXGazjL1XW2S8yYLLld9LTtt/dOXu7S C5/13npHOyl6wSdnCxivgcNWtXTdm37nJKb81wypASuCw4vdrFYuyZymzPv5aKif2ts7p5Ok+ Hf2jGb3lrxWuHUxZs+bu1ntenWf698eyzf7h1vNv9srXEfOEDc20MsRFOEReP/9m3nF976Y9a XE+np+VWIozEg21mIuKEwEU6xK2vAIAAA== X-Env-Sender: prvs=267e7865b=roger.pau@citrix.com X-Msg-Ref: server-2.tower-21.messagelabs.com!1491905156!49281581!1 X-Originating-IP: [66.165.176.89] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni44OSA9PiAyMDMwMDc=\n, received_headers: No Received headers X-StarScan-Received: X-StarScan-Version: 9.4.12; banners=-,-,- X-VirusChecked: Checked Received: (qmail 54181 invoked from network); 11 Apr 2017 10:05:58 -0000 Received: from smtp.citrix.com (HELO SMTP.CITRIX.COM) (66.165.176.89) by server-2.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 11 Apr 2017 10:05:58 -0000 X-IronPort-AV: E=Sophos;i="5.37,185,1488844800"; d="scan'208";a="418561191" From: Roger Pau Monne To: Date: Tue, 11 Apr 2017 11:04:01 +0100 Message-ID: <20170411100402.56246-6-roger.pau@citrix.com> X-Mailer: git-send-email 2.11.0 (Apple Git-81) In-Reply-To: <20170411100402.56246-1-roger.pau@citrix.com> References: <20170411100402.56246-1-roger.pau@citrix.com> MIME-Version: 1.0 Cc: Stefano Stabellini , Wei Liu , George Dunlap , Andrew Cooper , Ian Jackson , Tim Deegan , julien.grall@arm.com, Jan Beulich , boris.ostrovsky@oracle.com, Roger Pau Monne Subject: [Xen-devel] [PATCH for-next 5/6] xen/vpci: add handlers to map the BARs X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Introduce a set of handlers that trap accesses to the PCI BARs and the command register, in order to emulate BAR sizing and BAR relocation. The command handler is used to detect changes to bit 2 (response to memory space accesses), and maps/unmaps the BARs of the device into the guest p2m. The BAR register handlers are used to detect attempts by the guest to size or relocate the BARs. Signed-off-by: Roger Pau Monné --- Cc: Andrew Cooper Cc: George Dunlap Cc: Ian Jackson Cc: Jan Beulich Cc: Konrad Rzeszutek Wilk Cc: Stefano Stabellini Cc: Tim Deegan Cc: Wei Liu --- xen/drivers/vpci/Makefile | 2 +- xen/drivers/vpci/header.c | 244 ++++++++++++++++++++++++++++++++++++++++++++++ xen/include/xen/vpci.h | 23 +++++ 3 files changed, 268 insertions(+), 1 deletion(-) create mode 100644 xen/drivers/vpci/header.c diff --git a/xen/drivers/vpci/Makefile b/xen/drivers/vpci/Makefile index 840a906470..241467212f 100644 --- a/xen/drivers/vpci/Makefile +++ b/xen/drivers/vpci/Makefile @@ -1 +1 @@ -obj-y += vpci.o +obj-y += vpci.o header.o diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c new file mode 100644 index 0000000000..a5f36ea4ab --- /dev/null +++ b/xen/drivers/vpci/header.c @@ -0,0 +1,244 @@ +/* + * Generic functionality for handling accesses to the PCI header from the + * configuration space. + * + * Copyright (C) 2017 Citrix Systems R&D + * + * This program is free software; you can redistribute it and/or + * modify it under the terms and conditions of the GNU General Public + * License, version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + * + * You should have received a copy of the GNU General Public + * License along with this program; If not, see . + */ + +#include +#include +#include + +static int vpci_cmd_read(struct pci_dev *pdev, unsigned int reg, + union vpci_val *val, void *data) +{ + uint16_t *cmd = data; + + val->word = *cmd; + return 0; +} + +static int vpci_cmd_write(struct pci_dev *pdev, unsigned int reg, + union vpci_val val, void *data) +{ + uint16_t new_cmd = val.word, *saved_cmd = data; + struct vpci_bar *bars = pdev->vpci->header.bars; + uint8_t seg = pdev->seg, bus = pdev->bus; + uint8_t slot = PCI_SLOT(pdev->devfn), func = PCI_FUNC(pdev->devfn); + unsigned int i; + int rc; + + + if ( !((new_cmd ^ *saved_cmd) & PCI_COMMAND_MEMORY) ) + goto out; + + /* Memory space access change. */ + for ( i = 0; i < ARRAY_SIZE(pdev->vpci->header.bars); i++ ) + { + bool map = !!(new_cmd & PCI_COMMAND_MEMORY); + + if ( bars[i].type != VPCI_BAR_MEM && + bars[i].type != VPCI_BAR_MEM64_LO ) + continue; + + if ( !map && !bars[i].mapped_addr ) + continue; + + rc = modify_identity_mmio(pdev->domain, PFN_DOWN(bars[i].addr), + PFN_UP(bars[i].size), map); + if ( rc ) + continue; + + bars[i].mapped_addr = map ? bars[i].addr : 0; + } + + out: + pci_conf_write16(seg, bus, slot, func, reg, new_cmd); + *saved_cmd = pci_conf_read16(seg, bus, slot, func, reg); + return 0; +} + +static int vpci_bar_read(struct pci_dev *pdev, unsigned int reg, + union vpci_val *val, void *data) +{ + struct vpci_bar *bar = data; + bool hi = false; + + ASSERT(bar->type == VPCI_BAR_MEM || bar->type == VPCI_BAR_MEM64_LO || + bar->type == VPCI_BAR_MEM64_HI); + + if ( bar->type == VPCI_BAR_MEM64_HI ) + { + ASSERT(reg - PCI_BASE_ADDRESS_0 > 0); + bar--; + hi = true; + } + + if ( bar->sizing ) + val->double_word = ~(bar->size - 1) >> (hi ? 32 : 0); + else + val->double_word = bar->addr >> (hi ? 32 : 0); + + val->double_word |= hi ? 0 : bar->attributes; + + return 0; +} + +static int vpci_bar_write(struct pci_dev *pdev, unsigned int reg, + union vpci_val val, void *data) +{ + struct vpci_bar *bar = data; + uint32_t wdata = val.double_word; + bool hi = false; + + ASSERT(bar->type == VPCI_BAR_MEM || bar->type == VPCI_BAR_MEM64_LO || + bar->type == VPCI_BAR_MEM64_HI); + + if ( wdata == GENMASK(31, 0) ) + { + /* Next reads from this register are going to return the BAR size. */ + bar->sizing = true; + return 0; + } + + /* End previous sizing cycle if any. */ + bar->sizing = false; + + if ( bar->type == VPCI_BAR_MEM64_HI ) + { + ASSERT(reg - PCI_BASE_ADDRESS_0 > 0); + bar--; + hi = true; + } + + /* Update the relevant part of the BAR address. */ + bar->addr &= hi ? ~GENMASK(63, 32) : ~GENMASK(31, 0); + wdata &= hi ? GENMASK(31, 0) : PCI_BASE_ADDRESS_MEM_MASK; + bar->addr |= (uint64_t)wdata << (hi ? 32 : 0); + + return 0; +} + +static int vpci_init_bars(struct pci_dev *pdev) +{ + uint8_t seg = pdev->seg, bus = pdev->bus; + uint8_t slot = PCI_SLOT(pdev->devfn), func = PCI_FUNC(pdev->devfn); + uint8_t header_type; + unsigned int i, num_bars; + struct vpci_header *header = &pdev->vpci->header; + struct vpci_bar *bars = header->bars; + int rc; + + header_type = pci_conf_read8(seg, bus, slot, func, PCI_HEADER_TYPE) & 0x7f; + if ( header_type == PCI_HEADER_TYPE_NORMAL ) + num_bars = 6; + else if ( header_type == PCI_HEADER_TYPE_BRIDGE ) + num_bars = 2; + else + return -ENOSYS; + + /* Setup a handler for the control register. */ + header->command = pci_conf_read16(seg, bus, slot, func, PCI_COMMAND); + rc = xen_vpci_add_register(pdev, vpci_cmd_read, vpci_cmd_write, + PCI_COMMAND, 2, &header->command); + if ( rc ) + { + dprintk(XENLOG_ERR, + "%04x:%02x:%02x.%u: failed to add handler register %#x: %d\n", + seg, bus, slot, func, PCI_COMMAND, rc); + return rc; + } + + + for ( i = 0; i < num_bars; i++ ) + { + uint8_t reg = PCI_BASE_ADDRESS_0 + i * 4; + uint32_t val = pci_conf_read32(seg, bus, slot, func, reg); + uint64_t addr, size; + unsigned int index; + + if ( i && bars[i - 1].type == VPCI_BAR_MEM64_LO ) + { + bars[i].type = VPCI_BAR_MEM64_HI; + continue; + } + else if ( (val & PCI_BASE_ADDRESS_SPACE) == PCI_BASE_ADDRESS_SPACE_IO ) + { + bars[i].type = VPCI_BAR_IO; + continue; + } + else if ( (val & PCI_BASE_ADDRESS_MEM_TYPE_MASK) == + PCI_BASE_ADDRESS_MEM_TYPE_64 ) + bars[i].type = VPCI_BAR_MEM64_LO; + else + bars[i].type = VPCI_BAR_MEM; + + /* Size the BAR and map it. */ + index = i; + rc = pci_size_bar(seg, bus, slot, func, PCI_BASE_ADDRESS_0, num_bars, + &index, &addr, &size); + if ( rc ) + { + dprintk(XENLOG_ERR, + "%04x:%02x:%02x.%u: unable to size BAR#%u: %d\n", + seg, bus, slot, func, i, rc); + return rc; + } + + if ( size == 0 ) + { + bars[i].type = VPCI_BAR_EMPTY; + continue; + } + + bars[i].addr = addr; + bars[i].size = size; + bars[i].attributes = val & ~PCI_BASE_ADDRESS_MEM_MASK; + + if ( header->command & PCI_COMMAND_MEMORY ) + { + /* Memory space accesses are active, map BARs. */ + rc = modify_identity_mmio(pdev->domain, PFN_DOWN(bars[i].addr), + PFN_UP(bars[i].size), true); + if ( !rc ) + bars[i].mapped_addr = addr; + } + + rc = xen_vpci_add_register(pdev, vpci_bar_read, vpci_bar_write, reg, + 4, &bars[i]); + if ( rc ) + { + dprintk(XENLOG_ERR, + "%04x:%02x:%02x.%u: failed to add handler for BAR#%u: %d\n", + seg, bus, slot, func, i, rc); + return rc; + } + } + + return 0; +} + +REGISTER_VPCI_INIT(vpci_init_bars); + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ + diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index 2f54919310..205d121a92 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -46,6 +46,29 @@ int xen_vpci_write(unsigned int seg, unsigned int bus, unsigned int devfn, struct vpci { /* Root pointer for the tree of vPCI handlers. */ struct rb_root handlers; + +#ifdef __XEN__ + /* Hide the rest of the vpci struct from the user-space test harness. */ + struct vpci_header { + /* Cached value of the command register. */ + uint16_t command; + /* Information about the PCI BARs of this device. */ + struct vpci_bar { + enum { + VPCI_BAR_EMPTY, + VPCI_BAR_IO, + VPCI_BAR_MEM, + VPCI_BAR_MEM64_LO, + VPCI_BAR_MEM64_HI, + } type; + paddr_t addr; + paddr_t mapped_addr; + size_t size; + unsigned int attributes:4; + bool sizing; + } bars[6]; + } header; +#endif }; #endif