From patchwork Wed Oct 31 13:29:38 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel J Blueman X-Patchwork-Id: 1678981 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id 4CDB8DF264 for ; Wed, 31 Oct 2012 13:31:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753879Ab2JaNbA (ORCPT ); Wed, 31 Oct 2012 09:31:00 -0400 Received: from mail-pa0-f46.google.com ([209.85.220.46]:56369 "EHLO mail-pa0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751187Ab2JaNa7 (ORCPT ); Wed, 31 Oct 2012 09:30:59 -0400 Received: by mail-pa0-f46.google.com with SMTP id hz1so965919pad.19 for ; Wed, 31 Oct 2012 06:30:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=from:to:cc:subject:date:message-id:x-mailer:x-gm-message-state; bh=dj5krajIqXTwL1Hh/9queFalixNgfBjnJvAy4y9hnPQ=; b=LgYDz88mzjCxmKdmV+UZpbho5IMeUHu6xqYV5amuabPUr+11YdsVc8Reh0JvFbWO4B 3axWtP7JW02y2XyHqedao195Sutj7ptqg1jCcVftBnA7tFo/aviYkLj/jBwdT1dauYcj CxiRI7Xy9vdCanaaZ5E7e0IyxKuzsi0pdB7WSIrjM5uHSBrGusBGDHva4lGqHemSq8b8 /w3de6vTpr5svv/GAk0A11scd458DmugHgPBrrjGXAVWPUhppJIcq1GtSRyDZtnBYvnX 9LrmgzXe5XGKBHKR/ryRWS7z/ohZ1yoEquqCo7wiV5wsRUGydPL9VxzYIBy25wn1d8P8 FioA== Received: by 10.68.225.5 with SMTP id rg5mr111424399pbc.73.1351690258733; Wed, 31 Oct 2012 06:30:58 -0700 (PDT) Received: from meiko.1dn ([115.42.131.38]) by mx.google.com with ESMTPS id ok8sm2305842pbb.42.2012.10.31.06.30.54 (version=SSLv3 cipher=OTHER); Wed, 31 Oct 2012 06:30:57 -0700 (PDT) From: Daniel J Blueman To: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Steffen Persvold , Bjorn Helgaas , Daniel J Blueman Subject: [PATCH v2] Add NumaChip remote PCI support Date: Wed, 31 Oct 2012 21:29:38 +0800 Message-Id: <1351690178-30448-1-git-send-email-daniel@numascale-asia.com> X-Mailer: git-send-email 1.7.9.5 X-Gm-Message-State: ALoCoQmsqdjaGIHwAcS11vMAMnrGwPQ0HdN2w8dXlsdUTEIMDxRHwwtYyYlLC0BZWsg4m933dwdA Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Add NumaChip-specific PCI access mechanism via MMCONFIG cycles, but preventing access to AMD Northbridges which shouldn't respond. v2: Use PCI_DEVFN in precomputed constant limit; drop unneeded includes Signed-off-by: Daniel J Blueman --- arch/x86/include/asm/numachip/numachip.h | 20 +++++ arch/x86/kernel/apic/apic_numachip.c | 2 + arch/x86/pci/Makefile | 1 + arch/x86/pci/numachip.c | 134 ++++++++++++++++++++++++++++++ 4 files changed, 157 insertions(+) create mode 100644 arch/x86/include/asm/numachip/numachip.h create mode 100644 arch/x86/pci/numachip.c diff --git a/arch/x86/include/asm/numachip/numachip.h b/arch/x86/include/asm/numachip/numachip.h new file mode 100644 index 0000000..d35e71a --- /dev/null +++ b/arch/x86/include/asm/numachip/numachip.h @@ -0,0 +1,20 @@ +/* + * This file is subject to the terms and conditions of the GNU General Public + * License. See the file "COPYING" in the main directory of this archive + * for more details. + * + * Numascale NumaConnect-specific header file + * + * Copyright (C) 2012 Numascale AS. All rights reserved. + * + * Send feedback to + * + */ + +#ifndef _ASM_X86_NUMACHIP_NUMACHIP_H +#define _ASM_X86_NUMACHIP_NUMACHIP_H + +extern int __init pci_numachip_init(void); + +#endif /* _ASM_X86_NUMACHIP_NUMACHIP_H */ + diff --git a/arch/x86/kernel/apic/apic_numachip.c b/arch/x86/kernel/apic/apic_numachip.c index a65829a..9c2aa89 100644 --- a/arch/x86/kernel/apic/apic_numachip.c +++ b/arch/x86/kernel/apic/apic_numachip.c @@ -22,6 +22,7 @@ #include #include +#include #include #include #include @@ -179,6 +180,7 @@ static int __init numachip_system_init(void) return 0; x86_cpuinit.fixup_cpu_id = fixup_cpu_id; + x86_init.pci.arch_init = pci_numachip_init; map_csrs(); diff --git a/arch/x86/pci/Makefile b/arch/x86/pci/Makefile index 3af5a1e..ee0af58 100644 --- a/arch/x86/pci/Makefile +++ b/arch/x86/pci/Makefile @@ -16,6 +16,7 @@ obj-$(CONFIG_STA2X11) += sta2x11-fixup.o obj-$(CONFIG_X86_VISWS) += visws.o obj-$(CONFIG_X86_NUMAQ) += numaq_32.o +obj-$(CONFIG_X86_NUMACHIP) += numachip.o obj-$(CONFIG_X86_INTEL_MID) += mrst.o diff --git a/arch/x86/pci/numachip.c b/arch/x86/pci/numachip.c new file mode 100644 index 0000000..3773e05 --- /dev/null +++ b/arch/x86/pci/numachip.c @@ -0,0 +1,129 @@ +/* + * This file is subject to the terms and conditions of the GNU General Public + * License. See the file "COPYING" in the main directory of this archive + * for more details. + * + * Numascale NumaConnect-specific PCI code + * + * Copyright (C) 2012 Numascale AS. All rights reserved. + * + * Send feedback to + * + * PCI accessor functions derived from mmconfig_64.c + * + */ + +#include +#include + +static u8 limit __read_mostly; + +static inline char __iomem *pci_dev_base(unsigned int seg, unsigned int bus, unsigned int devfn) +{ + struct pci_mmcfg_region *cfg = pci_mmconfig_lookup(seg, bus); + + if (cfg && cfg->virt) + return cfg->virt + (PCI_MMCFG_BUS_OFFSET(bus) | (devfn << 12)); + return NULL; +} + +static int pci_mmcfg_read_numachip(unsigned int seg, unsigned int bus, + unsigned int devfn, int reg, int len, u32 *value) +{ + char __iomem *addr; + + /* Why do we have this when nobody checks it. How about a BUG()!? -AK */ + if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095))) { +err: *value = -1; + return -EINVAL; + } + + /* Ensure AMD Northbridges don't decode reads to other devices */ + if (unlikely(bus == 0 && devfn >= limit)) { + *value = -1; + return 0; + } + + rcu_read_lock(); + addr = pci_dev_base(seg, bus, devfn); + if (!addr) { + rcu_read_unlock(); + goto err; + } + + switch (len) { + case 1: + *value = mmio_config_readb(addr + reg); + break; + case 2: + *value = mmio_config_readw(addr + reg); + break; + case 4: + *value = mmio_config_readl(addr + reg); + break; + } + rcu_read_unlock(); + + return 0; +} + +static int pci_mmcfg_write_numachip(unsigned int seg, unsigned int bus, + unsigned int devfn, int reg, int len, u32 value) +{ + char __iomem *addr; + + /* Why do we have this when nobody checks it. How about a BUG()!? -AK */ + if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095))) + return -EINVAL; + + /* Ensure AMD Northbridges don't decode writes to other devices */ + if (unlikely(bus == 0 && devfn >= limit)) + return 0; + + rcu_read_lock(); + addr = pci_dev_base(seg, bus, devfn); + if (!addr) { + rcu_read_unlock(); + return -EINVAL; + } + + switch (len) { + case 1: + mmio_config_writeb(addr + reg, value); + break; + case 2: + mmio_config_writew(addr + reg, value); + break; + case 4: + mmio_config_writel(addr + reg, value); + break; + } + rcu_read_unlock(); + + return 0; +} + +const struct pci_raw_ops pci_mmcfg_numachip = { + .read = pci_mmcfg_read_numachip, + .write = pci_mmcfg_write_numachip, +}; + +int __init pci_numachip_init(void) +{ + int ret = 0; + u32 val; + + /* For remote I/O, restrict bus 0 access to the actual number of AMD + Northbridges, which starts at device number 0x18 */ + ret = raw_pci_read(0, 0, PCI_DEVFN(0x18, 0), 0x60, sizeof(val), &val); + if (ret) + goto out; + + /* HyperTransport fabric size in bits 6:4 */ + limit = PCI_DEVFN(0x18 + ((val >> 4) & 7) + 1, 0); + + /* Use NumaChip PCI accessors for non-extended and extended access */ + raw_pci_ops = raw_pci_ext_ops = &pci_mmcfg_numachip; +out: + return ret; +}