From patchwork Fri Feb 5 23:41:15 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Daney X-Patchwork-Id: 8240581 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id E72D79F3CD for ; Fri, 5 Feb 2016 23:43:47 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 771E7203C0 for ; Fri, 5 Feb 2016 23:43:46 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0A75D202E5 for ; Fri, 5 Feb 2016 23:43:45 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1aRq0y-0006iz-G9; Fri, 05 Feb 2016 23:42:12 +0000 Received: from mail-pa0-x243.google.com ([2607:f8b0:400e:c03::243]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1aRq0Y-0006CM-Ps for linux-arm-kernel@lists.infradead.org; Fri, 05 Feb 2016 23:41:52 +0000 Received: by mail-pa0-x243.google.com with SMTP id pv5so3011838pac.0 for ; Fri, 05 Feb 2016 15:41:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=tN4KHM7exp0xH/RcTofWeAMHgrZq2KcbItpVWPYRSM8=; b=iR0R+7+QT2Lh61vF7tHoxOlO0oDbRgcNRQJ0fIdweiit83+Oib2YkKYwNCaMmqgyn/ GqVOv4LizwJnvzLY/1N84pe/7AoxeelimEeixK9V8WmmBTqzfrYIQlTD2H3xZXsD0kkz AZq2J9yOU1cSqY+UgYb7Qvy6z1OaUfDcC73DD50vATzF0XTsq2FgE73O3OaUODrk0PUT Le1FEPAbsJ4kHx4TexIGgkcMIIOm+ooXFAn1u515Y0J21YPobfg2R4y/GO/9BSm3IEZT U7Lh6IhgM90TSjGk+gAxBvnzFco8UHQp8nHXCcacLgxEVJK/1qBYyvi/v7E45mhLG9+g l7vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=tN4KHM7exp0xH/RcTofWeAMHgrZq2KcbItpVWPYRSM8=; b=Gu4Ced/z0h49lZ3iy4759AEYxSFHYCYHlioBax5n3evT3mmeHIFORn+Ey+VUhG3P4k vmRX22niHmeFuagxsRHoGTvZDR0E5zkBdqvTxQcwbJ4eR6cgMn4Jzi4pIIZ6CM6fObKX 6B16uOblIDS6iLUPLINSD3GsdtMuuIP9deRUJsuw0UvnczgM6Yy7UvU0/fmhdUm52yGs 9WKUe+GRFsw89y4uJEQm2B5Lv+lQkIgnhIL7R9daiqJeRDuqBzwo6tLYOTqavYllwTGB wZ5O8gVXRIn6vHM6oQctmN5f72YDv24M0lc41jB3XzdD222tXl2aevw+izQN0wHZZa/8 glhA== X-Gm-Message-State: AG10YOTF96BR1zP56zDoIY6Gyfxsp4r4IA/mFE5m4DEB3BjnxSKs5qnhIOL52m6Lpz/qHg== X-Received: by 10.66.193.195 with SMTP id hq3mr23925694pac.38.1454715686088; Fri, 05 Feb 2016 15:41:26 -0800 (PST) Received: from dl.caveonetworks.com ([64.2.3.194]) by smtp.gmail.com with ESMTPSA id o17sm26847034pfj.50.2016.02.05.15.41.19 (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 05 Feb 2016 15:41:22 -0800 (PST) Received: from dl.caveonetworks.com (localhost.localdomain [127.0.0.1]) by dl.caveonetworks.com (8.14.5/8.14.5) with ESMTP id u15NfIPR017560; Fri, 5 Feb 2016 15:41:18 -0800 Received: (from ddaney@localhost) by dl.caveonetworks.com (8.14.5/8.14.5/Submit) id u15NfIeG017559; Fri, 5 Feb 2016 15:41:18 -0800 From: David Daney To: Bjorn Helgaas , linux-pci@vger.kernel.org, Will Deacon , linux-arm-kernel@lists.infradead.org, Rob Herring , Pawel Moll , Mark Rutland , Ian Campbell , Kumar Gala , devicetree@vger.kernel.org Subject: [PATCH v5 3/3] pci, pci-thunder-ecam: Add driver for ThunderX-pass1 on-chip devices Date: Fri, 5 Feb 2016 15:41:15 -0800 Message-Id: <1454715675-17512-4-git-send-email-ddaney.cavm@gmail.com> X-Mailer: git-send-email 1.7.11.7 In-Reply-To: <1454715675-17512-1-git-send-email-ddaney.cavm@gmail.com> References: <1454715675-17512-1-git-send-email-ddaney.cavm@gmail.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160205_154147_097464_FE86C0D3 X-CRM114-Status: GOOD ( 27.24 ) X-Spam-Score: -2.7 (--) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-kernel@vger.kernel.org, David Daney MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: David Daney The cavium,pci-thunder-ecam devices are exactly ECAM based PCI root complexes. These root complexes (loosely referred to as ECAM units in the hardware manuals) are used to access the Thunder on-chips devices. They are special in that all the BARs on devices behind these root complexes are at fixed addresses. To handle this in a manner compatible with the core PCI code, we have the config access functions synthesize Enhanced Allocation (EA) capability entries for each BAR. Since this EA synthesis is needed for exactly one chip model, we can hard code some assumptions about the device topology and the properties of specific DEVFNs in the driver. Signed-off-by: David Daney Acked-by: Rob Herring --- .../devicetree/bindings/pci/pci-thunder-ecam.txt | 30 ++ drivers/pci/host/Kconfig | 7 + drivers/pci/host/Makefile | 1 + drivers/pci/host/pci-thunder-ecam.c | 358 +++++++++++++++++++++ 4 files changed, 396 insertions(+) create mode 100644 Documentation/devicetree/bindings/pci/pci-thunder-ecam.txt create mode 100644 drivers/pci/host/pci-thunder-ecam.c diff --git a/Documentation/devicetree/bindings/pci/pci-thunder-ecam.txt b/Documentation/devicetree/bindings/pci/pci-thunder-ecam.txt new file mode 100644 index 0000000..34658f2 --- /dev/null +++ b/Documentation/devicetree/bindings/pci/pci-thunder-ecam.txt @@ -0,0 +1,30 @@ +* ThunderX PCI host controller for pass-1.x silicon + +Firmware-initialized PCI host controller to on-chip devices found on +some Cavium ThunderX processors. These devices have ECAM based config +access, but the BARs are all at fixed addresses. We handle the fixed +addresses by synthesizing Enhanced Allocation (EA) capabilities for +these devices. + +The properties and their meanings are identical to those described in +host-generic-pci.txt except as listed below. + +Properties of the host controller node that differ from +host-generic-pci.txt: + +- compatible : Must be "cavium,pci-host-thunder-ecam" + +Example: + + pci@84b0,00000000 { + compatible = "cavium,pci-host-thunder-ecam"; + device_type = "pci"; + msi-parent = <&its>; + msi-map = <0 &its 0x30000 0x10000>; + bus-range = <0 31>; + #size-cells = <2>; + #address-cells = <3>; + #stream-id-cells = <1>; + reg = <0x84b0 0x00000000 0 0x02000000>; /* Configuration space */ + ranges = <0x03000000 0x8180 0x00000000 0x8180 0x00000000 0x80 0x00000000>; /* mem ranges */ + }; diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig index 184df22..f8912c6 100644 --- a/drivers/pci/host/Kconfig +++ b/drivers/pci/host/Kconfig @@ -202,4 +202,11 @@ config PCI_HOST_THUNDER_PEM help Say Y here if you want PCIe support for CN88XX Cavium Thunder SoCs. +config PCI_HOST_THUNDER_ECAM + bool "Cavium Thunder ECAMe controller to on-chip devices on pass-1.x silicon" + depends on OF && ARM64 + select PCI_HOST_COMMON + help + Say Y here if you want ECAM support for CN88XX-Pass-1.x Cavium Thunder SoCs. + endmenu diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile index 8903172..d6af3ba 100644 --- a/drivers/pci/host/Makefile +++ b/drivers/pci/host/Makefile @@ -23,4 +23,5 @@ obj-$(CONFIG_PCIE_ALTERA) += pcie-altera.o obj-$(CONFIG_PCIE_ALTERA_MSI) += pcie-altera-msi.o obj-$(CONFIG_PCI_HISI) += pcie-hisi.o obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o +obj-$(CONFIG_PCI_HOST_THUNDER_ECAM) += pci-thunder-ecam.o obj-$(CONFIG_PCI_HOST_THUNDER_PEM) += pci-thunder-pem.o diff --git a/drivers/pci/host/pci-thunder-ecam.c b/drivers/pci/host/pci-thunder-ecam.c new file mode 100644 index 0000000..83ee590 --- /dev/null +++ b/drivers/pci/host/pci-thunder-ecam.c @@ -0,0 +1,358 @@ +/* + * + * This file is subject to the terms and conditions of the GNU General Public + * License. See the file "COPYING" in the main directory of this archive + * for more details. + * + * Copyright (C) 2015 Cavium, Inc. + * + */ + +#include +#include +#include +#include +#include +#include + +#include "pci-host-common.h" + +/* Mapping is standard ECAM */ +static void __iomem *thunder_ecam_map_bus(struct pci_bus *bus, + unsigned int devfn, + int where) +{ + struct gen_pci *pci = bus->sysdata; + resource_size_t idx = bus->number - pci->cfg.bus_range->start; + + return pci->cfg.win[idx] + ((devfn << 12) | where); +} + +static void set_val(u32 v, int where, int size, u32 *val) +{ + int shift = (where & 3) * 8; + + pr_debug("set_val %04x: %08x\n", (unsigned)(where & ~3), v); + v >>= shift; + if (size == 1) + v &= 0xff; + else if (size == 2) + v &= 0xffff; + *val = v; +} + +static int handle_ea_bar(u32 e0, int bar, struct pci_bus *bus, + unsigned int devfn, int where, int size, u32 *val) +{ + void __iomem *addr; + u32 v; + /* + * Each entry is 16-byte aligned bits[2,3] select which word + * in the entry + */ + int where_a = where & 0xc; + + if (where_a == 0) { + set_val(e0, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + if (where_a == 0x4) { + addr = bus->ops->map_bus(bus, devfn, bar); /* BAR 0 */ + if (!addr) { + *val = ~0; + return PCIBIOS_DEVICE_NOT_FOUND; + } + v = readl(addr); + v &= ~0xf; + v |= 2; /* EA entry-1. Base-L */ + set_val(v, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + if (where_a == 0x8) { + u32 barl_orig; + u32 barl_rb; + + addr = bus->ops->map_bus(bus, devfn, bar); /* BAR 0 */ + if (!addr) { + *val = ~0; + return PCIBIOS_DEVICE_NOT_FOUND; + } + barl_orig = readl(addr + 0); + writel(0xffffffff, addr + 0); + barl_rb = readl(addr + 0); + writel(barl_orig, addr + 0); + /* zeros in unsettable bits. */ + v = ~barl_rb & ~3; + v |= 0xc; /* EA entry-2. Offset-L */ + set_val(v, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + if (where_a == 0xc) { + addr = bus->ops->map_bus(bus, devfn, bar + 4); /* BAR 1 */ + if (!addr) { + *val = ~0; + return PCIBIOS_DEVICE_NOT_FOUND; + } + v = readl(addr); /* EA entry-3. Base-H */ + set_val(v, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + return PCIBIOS_DEVICE_NOT_FOUND; +} + +static int thunder_ecam_config_read(struct pci_bus *bus, unsigned int devfn, + int where, int size, u32 *val) +{ + u32 v; + u32 vendor_device; + void __iomem *addr; + int cfg_type; + int where_a = where & ~3; + + + addr = bus->ops->map_bus(bus, devfn, 0xc); + if (!addr) { + *val = ~0; + return PCIBIOS_DEVICE_NOT_FOUND; + } + + v = readl(addr); + + /* Check for non type-00 header. */ + cfg_type = (v >> 16) & 0x7f; + + /* + * All BARs have fixed addresses specified by the EA + * capability, they must return zero on read. + */ + if (cfg_type == 0 && + ((where >= 0x10 && where < 0x2c) || + (where >= 0x1a4 && where < 0x1bc))) { + /* BAR or SRIOV BAR */ + *val = 0; + return PCIBIOS_SUCCESSFUL; + } + + addr = bus->ops->map_bus(bus, devfn, 0); + if (!addr) { + *val = ~0; + return PCIBIOS_DEVICE_NOT_FOUND; + } + + vendor_device = readl(addr); + if (vendor_device == 0xffffffff) + goto no_emulation; + + addr = bus->ops->map_bus(bus, devfn, 8); + if (!addr) { + *val = ~0; + return PCIBIOS_DEVICE_NOT_FOUND; + } + + v = readl(addr); + if (v == 0xffffffff) + goto no_emulation; + + pr_debug("%04x:%04x - Fix pass#: %08x, where: %03x, devfn: %03x\n", + vendor_device & 0xffff, vendor_device >> 16, v, + (unsigned) where, devfn); + + /* Check for non type-00 header. */ + if (cfg_type == 0) { + bool has_msix; + bool is_nic = (vendor_device == 0xa01e177d); + bool is_tns = (vendor_device == 0xa01f177d); + + addr = bus->ops->map_bus(bus, devfn, 0x70); + if (!addr) { + *val = ~0; + return PCIBIOS_DEVICE_NOT_FOUND; + } + /* E_CAP */ + v = readl(addr); + has_msix = (v & 0xff00) != 0; + + if (!has_msix && where_a == 0x70) { + v |= 0xbc00; /* next capability is EA at 0xbc */ + set_val(v, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + if (where_a == 0xb0) { + addr = bus->ops->map_bus(bus, devfn, where_a); + if (!addr) { + *val = ~0; + return PCIBIOS_DEVICE_NOT_FOUND; + } + v = readl(addr); + if (v & 0xff00) + pr_err("Bad MSIX cap header: %08x\n", v); + v |= 0xbc00; /* next capability is EA at 0xbc */ + set_val(v, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + if (where_a == 0xbc) { + if (is_nic) + v = 0x40014; /* EA last in chain, 4 entries. */ + else if (is_tns) + v = 0x30014; /* EA last in chain, 3 entries. */ + else if (has_msix) + v = 0x20014; /* EA last in chain, 2 entries. */ + else + v = 0x10014; /* EA last in chain, 1 entry. */ + set_val(v, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + if (where_a >= 0xc0 && where_a < 0xd0) + /* EA entry-0. PP=0, BAR0 Size:3 */ + return handle_ea_bar(0x80ff0003, + 0x10, bus, devfn, where, + size, val); + if (where_a >= 0xd0 && where_a < 0xe0 && has_msix) + /* EA entry-1. PP=0, BAR4 Size:3 */ + return handle_ea_bar(0x80ff0043, + 0x20, bus, devfn, where, + size, val); + if (where_a >= 0xe0 && where_a < 0xf0 && is_tns) + /* EA entry-2. PP=0, BAR2, Size:3 */ + return handle_ea_bar(0x80ff0023, + 0x18, bus, devfn, where, + size, val); + if (where_a >= 0xe0 && where_a < 0xf0 && is_nic) + /* EA entry-2. PP=4, VF_BAR0 (9), Size:3 */ + return handle_ea_bar(0x80ff0493, + 0x1a4, bus, devfn, where, + size, val); + if (where_a >= 0xf0 && where_a < 0x100 && is_nic) + /* EA entry-3. PP=4, VF_BAR4 (d), Size:3 */ + return handle_ea_bar(0x80ff04d3, + 0x1b4, bus, devfn, where, + size, val); + } else if (cfg_type == 1) { + bool is_rsl_bridge = devfn == 0x08; + bool is_rad_bridge = devfn == 0xa0; + bool is_zip_bridge = devfn == 0xa8; + bool is_dfa_bridge = devfn == 0xb0; + bool is_nic_bridge = devfn == 0x10; + + if (where_a == 0x70) { + addr = bus->ops->map_bus(bus, devfn, where_a); + if (!addr) { + *val = ~0; + return PCIBIOS_DEVICE_NOT_FOUND; + } + v = readl(addr); + if (v & 0xff00) + pr_err("Bad PCIe cap header: %08x\n", v); + v |= 0xbc00; /* next capability is EA at 0xbc */ + set_val(v, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + if (where_a == 0xbc) { + if (is_nic_bridge) + v = 0x10014; /* EA last in chain, 1 entry */ + else + v = 0x00014; /* EA last in chain, no entries */ + set_val(v, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + if (where_a == 0xc0) { + if (is_rsl_bridge || is_nic_bridge) + v = 0x0101; /* subordinate:secondary = 1:1 */ + else if (is_rad_bridge) + v = 0x0202; /* subordinate:secondary = 2:2 */ + else if (is_zip_bridge) + v = 0x0303; /* subordinate:secondary = 3:3 */ + else if (is_dfa_bridge) + v = 0x0404; /* subordinate:secondary = 4:4 */ + set_val(v, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + if (where_a == 0xc4 && is_nic_bridge) { + /* Enabled, not-Write, SP=ff, PP=05, BEI=6, ES=4 */ + v = 0x80ff0564; + set_val(v, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + if (where_a == 0xc8 && is_nic_bridge) { + v = 0x00000002; /* Base-L 64-bit */ + set_val(v, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + if (where_a == 0xcc && is_nic_bridge) { + v = 0xfffffffe; /* MaxOffset-L 64-bit */ + set_val(v, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + if (where_a == 0xd0 && is_nic_bridge) { + v = 0x00008430; /* NIC Base-H */ + set_val(v, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + if (where_a == 0xd4 && is_nic_bridge) { + v = 0x0000000f; /* MaxOffset-H */ + set_val(v, where, size, val); + return PCIBIOS_SUCCESSFUL; + } + } +no_emulation: + return pci_generic_config_read(bus, devfn, where, size, val); +} + +static int thunder_ecam_config_write(struct pci_bus *bus, unsigned int devfn, + int where, int size, u32 val) +{ + /* + * All BARs have fixed addresses, ignore BAR writes so they + * don't get corrupted. + */ + if ((where >= 0x10 && where < 0x2c) || + (where >= 0x1a4 && where < 0x1bc)) + /* BAR or SRIOV BAR */ + return PCIBIOS_SUCCESSFUL; + + return pci_generic_config_write(bus, devfn, where, size, val); +} + +static struct gen_pci_cfg_bus_ops thunder_ecam_bus_ops = { + .bus_shift = 20, + .ops = { + .map_bus = thunder_ecam_map_bus, + .read = thunder_ecam_config_read, + .write = thunder_ecam_config_write, + } +}; + +static const struct of_device_id thunder_ecam_of_match[] = { + { .compatible = "cavium,pci-host-thunder-ecam", + .data = &thunder_ecam_bus_ops }, + + { }, +}; +MODULE_DEVICE_TABLE(of, thunder_ecam_of_match); + +static int thunder_ecam_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + const struct of_device_id *of_id; + struct gen_pci *pci = devm_kzalloc(dev, sizeof(*pci), GFP_KERNEL); + + if (!pci) + return -ENOMEM; + + of_id = of_match_node(thunder_ecam_of_match, dev->of_node); + pci->cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data; + + return pci_host_common_probe(pdev, pci); +} + +static struct platform_driver thunder_ecam_driver = { + .driver = { + .name = KBUILD_MODNAME, + .of_match_table = thunder_ecam_of_match, + }, + .probe = thunder_ecam_probe, +}; +module_platform_driver(thunder_ecam_driver); + +MODULE_DESCRIPTION("Thunder ECAM PCI host driver"); +MODULE_LICENSE("GPL v2");