From patchwork Thu Dec 19 20:44:02 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yinghai Lu X-Patchwork-Id: 3384341 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 8508BC0D4A for ; Thu, 19 Dec 2013 20:45:41 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id ABA70206B9 for ; Thu, 19 Dec 2013 20:45:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6EDC3206AD for ; Thu, 19 Dec 2013 20:45:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756244Ab3LSUnz (ORCPT ); Thu, 19 Dec 2013 15:43:55 -0500 Received: from userp1040.oracle.com ([156.151.31.81]:31613 "EHLO userp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756038Ab3LSUnx (ORCPT ); Thu, 19 Dec 2013 15:43:53 -0500 Received: from acsinet21.oracle.com (acsinet21.oracle.com [141.146.126.237]) by userp1040.oracle.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.1) with ESMTP id rBJKhmNA020787 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 19 Dec 2013 20:43:49 GMT Received: from userz7021.oracle.com (userz7021.oracle.com [156.151.31.85]) by acsinet21.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id rBJKhltW015762 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 19 Dec 2013 20:43:47 GMT Received: from abhmp0016.oracle.com (abhmp0016.oracle.com [141.146.116.22]) by userz7021.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id rBJKhkD4017881; Thu, 19 Dec 2013 20:43:46 GMT Received: from linux-siqj.site (/10.132.126.191) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 19 Dec 2013 12:43:46 -0800 From: Yinghai Lu To: Bjorn Helgaas Cc: Guo Chao , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Yinghai Lu Subject: [PATCH v6 1/2] PCI: Try to allocate mem64 above 4G at first Date: Thu, 19 Dec 2013 12:44:02 -0800 Message-Id: <1387485843-17403-2-git-send-email-yinghai@kernel.org> X-Mailer: git-send-email 1.8.4 In-Reply-To: <1387485843-17403-1-git-send-email-yinghai@kernel.org> References: <1387485843-17403-1-git-send-email-yinghai@kernel.org> X-Source-IP: acsinet21.oracle.com [141.146.126.237] Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-7.4 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On system with more pcie cards, we do not have enough range under 4G to allocate those pci devices. On 64bit system, we could try to allocate mem64 above 4G at first, and fall back to below 4g if it can not find any above 4g. -v2: update bottom assigning to make it clear for non-pae support machine. -v3: Bjorn's change: use MAX_RESOURCE instead of -1 use start/end instead of bottom/max for all arch instead of just x86_64 -v4: updated after PCI_MAX_RESOURCE_32 change. -v5: restore io handling to use PCI_MAX_RESOURCE_32 as limit. -v6: checking pcibios_resource_to_bus return for every bus res, to decide it if we need to try high at first. It supports all arches instead of just x86_64. -v7: split 4G limit change out to another patch according to Bjorn. also use pci_clip_resource instead. -v8: refresh after changes in pci/resource. -v9: make second try to restart from first res of bus. so we can ommit the patch that sort resource list of pci root bus. Signed-off-by: Yinghai Lu --- drivers/pci/bus.c | 34 ++++++++++++++++++++++++---------- 1 file changed, 24 insertions(+), 10 deletions(-) diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c index 263b90c..d49e6cb 100644 --- a/drivers/pci/bus.c +++ b/drivers/pci/bus.c @@ -100,6 +100,9 @@ void pci_bus_remove_resources(struct pci_bus *bus) /* The region that can be mapped by a 32-bit BAR. */ static struct pci_bus_region pci_32_bit = {0, 0xffffffff}; +/* The region that can be mapped by a 64-bit BAR above 4G */ +static struct pci_bus_region pci_64_bit = {(resource_size_t)(1ULL<<32), + (resource_size_t)(-1ULL)}; /* * @res contains CPU addresses. Clip it so the corresponding bus addresses @@ -150,10 +153,11 @@ pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res, { int i, ret = -ENOMEM; struct resource *r; - resource_size_t max; + bool try_again = !!(res->flags & IORESOURCE_MEM_64); type_mask |= IORESOURCE_IO | IORESOURCE_MEM; +again: pci_bus_for_each_resource(bus, r, i) { struct resource avail; @@ -170,13 +174,21 @@ pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res, !(res->flags & IORESOURCE_PREFETCH)) continue; + /* If this is a 64-bit BAR, try above 4G first. */ + avail = *r; + if (try_again) { + /* res->flags has IORESOURCE_MEM_64 set */ + pci_clip_resource_to_bus(bus, &avail, &pci_64_bit); + if (!resource_size(&avail)) + continue; + } + /* * Unless this is a 64-bit BAR, we have to clip the * available space to the part that maps to the region of * 32-bit bus addresses. */ - avail = *r; - if (!(res->flags & IORESOURCE_MEM_64)) { + if (!try_again && !(res->flags & IORESOURCE_MEM_64)) { pci_clip_resource_to_bus(bus, &avail, &pci_32_bit); if (!resource_size(&avail)) continue; @@ -188,17 +200,19 @@ pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res, * this is an already-configured bridge window, its start * overrides "min". */ - if (avail.start) - min = avail.start; - - max = avail.end; /* Ok, try it out.. */ - ret = allocate_resource(r, res, size, min, max, - align, alignf, alignf_data); + ret = allocate_resource(r, res, size, avail.start ? : min, + avail.end, align, alignf, alignf_data); if (ret == 0) - break; + return 0; } + + if (try_again) { + try_again = false; + goto again; + } + return ret; }