From patchwork Wed May 18 01:48:00 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Guilherme G. Piccoli" X-Patchwork-Id: 9115951 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 1A433BF29F for ; Wed, 18 May 2016 01:57:40 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id BBE9520254 for ; Wed, 18 May 2016 01:57:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 98FE32022D for ; Wed, 18 May 2016 01:57:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752743AbcERB5g (ORCPT ); Tue, 17 May 2016 21:57:36 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:9155 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752499AbcERB5g (ORCPT ); Tue, 17 May 2016 21:57:36 -0400 X-Greylist: delayed 548 seconds by postgrey-1.27 at vger.kernel.org; Tue, 17 May 2016 21:57:35 EDT Received: from pps.filterd (m0048817.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.11/8.16.0.11) with SMTP id u4I1jpcZ000758 for ; Tue, 17 May 2016 21:48:26 -0400 Message-Id: <201605180148.u4I1jpcZ000758@mx0a-001b2d01.pphosted.com> Received: from e24smtp02.br.ibm.com (e24smtp02.br.ibm.com [32.104.18.86]) by mx0a-001b2d01.pphosted.com with ESMTP id 230akd0qut-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 17 May 2016 21:48:26 -0400 Received: from localhost by e24smtp02.br.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 17 May 2016 22:48:24 -0300 Received: from d24dlp02.br.ibm.com (9.18.248.206) by e24smtp02.br.ibm.com (10.172.0.142) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 17 May 2016 22:48:22 -0300 X-IBM-Helo: d24dlp02.br.ibm.com X-IBM-MailFrom: gpiccoli@linux.vnet.ibm.com X-IBM-RcptTo: linux-pci@vger.kernel.org Received: from d24relay03.br.ibm.com (d24relay03.br.ibm.com [9.13.184.25]) by d24dlp02.br.ibm.com (Postfix) with ESMTP id 3A6BF1DC0054 for ; Tue, 17 May 2016 21:48:15 -0400 (EDT) Received: from d24av03.br.ibm.com (d24av03.br.ibm.com [9.8.31.95]) by d24relay03.br.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id u4I1mL4T7078246 for ; Tue, 17 May 2016 22:48:21 -0300 Received: from d24av03.br.ibm.com (localhost [127.0.0.1]) by d24av03.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id u4I1mKQI023695 for ; Tue, 17 May 2016 22:48:21 -0300 Received: from localhost ([9.78.132.121]) by d24av03.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id u4I1mJvq023689; Tue, 17 May 2016 22:48:20 -0300 From: "Guilherme G. Piccoli" To: linuxppc-dev@lists.ozlabs.org, mpe@ellerman.id.au, benh@kernel.crashing.org, paulus@samba.org Cc: linux-pci@vger.kernel.org, imunsie@au1.ibm.com, mikey@neuling.org, andrew.donnellan@au1.ibm.com, gwshan@linux.vnet.ibm.com, bhelgaas@google.com, gpiccoli@linux.vnet.ibm.com, frederic.barrat@fr.ibm.com, mrochs@linux.vnet.ibm.com Subject: [PATCH v6] powerpc/pci: Assign fixed PHB number based on device-tree properties Date: Tue, 17 May 2016 22:48:00 -0300 X-Mailer: git-send-email 2.1.0 X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16051801-0020-0000-0000-000002042B05 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16051801-0021-0000-0000-00002FB4CA8E X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-05-17_09:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_spam_policy_notspam policy=outbound_spam_policy score=0 spamscore=0 suspectscore=25 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1604210000 definitions=main-1605180021 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-8.3 required=5.0 tests=BAYES_00, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The domain/PHB field of PCI addresses has its value obtained from a global variable, incremented each time a new domain (represented by struct pci_controller) is added on the system. The domain addition process happens during boot or due to PHB hotplug add. As recent kernels are using predictable naming for network interfaces, the network stack is more tied to PCI naming. This can be a problem in hotplug scenarios, because PCI addresses will change if devices are removed and then re-added. This situation seems unusual, but it can happen if a user wants to replace a NIC without rebooting the machine, for example. This patch changes the way PCI domain values are generated: now, we use device-tree properties to assign fixed PHB numbers to PCI addresses when available (meaning pSeries and PowerNV cases). We also use a bitmap to allow dynamic PHB numbering when device-tree properties are not used. This bitmap keeps track of used PHB numbers and if a PHB is released (by hotplug operations for example), it allows the reuse of this PHB number, avoiding PCI address to change in case of device remove and re-add soon after. No functional changes were introduced. Signed-off-by: Guilherme G. Piccoli Reviewed-by: Gavin Shan Reviewed-by: Ian Munsie --- v6: * Dropped the of_get_property() use to use instead of_property_read_u64()/of_property_read_u32_array as per Michael's suggestion. Since these 2 functions deal with endianness conversion, no need to manually convert endianness anymore. Logic was simplified. * Changed MAX_PHBS to 64K as per Gavin's suggestion. Changed 0xFFFF to (MAX_PHBS - 1) then. * Changed test_bit() and set_bit() to test_and_set_bit() as per Gavin's suggestion. * Removed some obvious comments. * Didn't remove machine check for pSeries on "reg" property lookup. It's worthy to keep it, since almost every platform (if not all of them) contain the "reg" property on PHB node in device-tree, but only in pSeries we're 100% sure it can be used as the PHB unique identifier. Since the patch has a dynamic PHB numbering mechanism, the other platforms won't have trouble with it. arch/powerpc/kernel/pci-common.c | 59 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 56 insertions(+), 3 deletions(-) diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c index 0f7a60f..4afc9c1 100644 --- a/arch/powerpc/kernel/pci-common.c +++ b/arch/powerpc/kernel/pci-common.c @@ -41,11 +41,18 @@ #include #include +/* hose_spinlock protects accesses to the the phb_bitmap. */ static DEFINE_SPINLOCK(hose_spinlock); LIST_HEAD(hose_list); -/* XXX kill that some day ... */ -static int global_phb_number; /* Global phb counter */ +/* For dynamic PHB numbering on get_phb_number(): max number of PHBs. */ +#define MAX_PHBS 0x10000 + +/* + * For dynamic PHB numbering: used/free PHBs tracking bitmap. + * Accesses to this bitmap should be protected by hose_spinlock. + */ +static DECLARE_BITMAP(phb_bitmap, MAX_PHBS); /* ISA Memory physical address */ resource_size_t isa_mem_base; @@ -64,6 +71,47 @@ struct dma_map_ops *get_pci_dma_ops(void) } EXPORT_SYMBOL(get_pci_dma_ops); +/* + * This function should run under locking protection, specifically + * hose_spinlock. + */ +static int get_phb_number(struct device_node *dn) +{ + u64 prop; + int ret; + int phb_id; + + /* + * Try fixed PHB numbering first, by checking archs and reading + * the respective device-tree properties. Firstly, try PowerNV by + * reading "ibm,opal-phbid", only present in OPAL environment. + */ + ret = of_property_read_u64(dn, "ibm,opal-phbid", &prop); + if (ret && machine_is(pseries)) + ret = of_property_read_u32_array(dn, "reg", (u32 *)&prop, 2); + if (ret) + goto dynamic_phb_numbering; + + phb_id = (int)(prop & (MAX_PHBS - 1)); + + /* We need to be sure to not use the same PHB number twice. */ + if (test_and_set_bit(phb_id, phb_bitmap)) + goto dynamic_phb_numbering; + + return phb_id; + + /* + * If not pSeries nor PowerNV, or if fixed PHB numbering tried to add + * the same PHB number twice, then fallback to dynamic PHB numbering. + */ +dynamic_phb_numbering: + phb_id = find_first_zero_bit(phb_bitmap, MAX_PHBS); + BUG_ON(phb_id >= MAX_PHBS); + set_bit(phb_id, phb_bitmap); + + return phb_id; +} + struct pci_controller *pcibios_alloc_controller(struct device_node *dev) { struct pci_controller *phb; @@ -72,7 +120,7 @@ struct pci_controller *pcibios_alloc_controller(struct device_node *dev) if (phb == NULL) return NULL; spin_lock(&hose_spinlock); - phb->global_number = global_phb_number++; + phb->global_number = get_phb_number(dev); list_add_tail(&phb->list_node, &hose_list); spin_unlock(&hose_spinlock); phb->dn = dev; @@ -94,6 +142,11 @@ EXPORT_SYMBOL_GPL(pcibios_alloc_controller); void pcibios_free_controller(struct pci_controller *phb) { spin_lock(&hose_spinlock); + + /* Clear bit of phb_bitmap to allow reuse of this PHB number. */ + if (phb->global_number < MAX_PHBS) + clear_bit(phb->global_number, phb_bitmap); + list_del(&phb->list_node); spin_unlock(&hose_spinlock);