From patchwork Fri Nov 14 03:59:27 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yinghai Lu X-Patchwork-Id: 5303041 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 88E6F9F2ED for ; Fri, 14 Nov 2014 03:59:41 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 70D72201C7 for ; Fri, 14 Nov 2014 03:59:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3DD51201BB for ; Fri, 14 Nov 2014 03:59:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935038AbaKND7h (ORCPT ); Thu, 13 Nov 2014 22:59:37 -0500 Received: from userp1040.oracle.com ([156.151.31.81]:42483 "EHLO userp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933512AbaKND7h (ORCPT ); Thu, 13 Nov 2014 22:59:37 -0500 Received: from ucsinet21.oracle.com (ucsinet21.oracle.com [156.151.31.93]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id sAE3xTr8010513 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 14 Nov 2014 03:59:30 GMT Received: from userz7021.oracle.com (userz7021.oracle.com [156.151.31.85]) by ucsinet21.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id sAE3xSwB019177 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Fri, 14 Nov 2014 03:59:28 GMT Received: from abhmp0020.oracle.com (abhmp0020.oracle.com [141.146.116.26]) by userz7021.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id sAE3xRqO019168; Fri, 14 Nov 2014 03:59:27 GMT Received: from linux-siqj.site (/107.215.0.145) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 13 Nov 2014 19:59:27 -0800 From: Yinghai Lu To: Bjorn Helgaas , Andrew Morton , "H. Peter Anvin" , Ingo Molnar Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Yinghai Lu Subject: [PATCH] x86, PCI: support mmio more than 44 bit on 32bit/PAE mode Date: Thu, 13 Nov 2014 19:59:27 -0800 Message-Id: <1415937567-14510-1-git-send-email-yinghai@kernel.org> X-Mailer: git-send-email 1.8.4.5 X-Source-IP: ucsinet21.oracle.com [156.151.31.93] Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-7.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Aaron reported 32bit/PAE mode, has problem with 64bit resource. [ 6.610012] pci 0000:03:00.0: reg 0x10: [mem 0x383fffc00000-0x383fffdfffff 64bit pref] [ 6.622195] pci 0000:03:00.0: reg 0x20: [mem 0x383fffe04000-0x383fffe07fff 64bit pref] [ 6.656112] pci 0000:03:00.1: reg 0x10: [mem 0x383fffa00000-0x383fffbfffff 64bit pref] [ 6.668293] pci 0000:03:00.1: reg 0x20: [mem 0x383fffe00000-0x383fffe03fff 64bit pref] ... [ 12.374143] calling ixgbe_init_module+0x0/0x51 @ 1 [ 12.378130] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 3.19.1-k [ 12.385318] ixgbe: Copyright (c) 1999-2014 Intel Corporation. [ 12.390578] ixgbe 0000:03:00.0: Adapter removed [ 12.394247] ixgbe: probe of 0000:03:00.0 failed with error -5 [ 12.399369] ixgbe 0000:03:00.1: Adapter removed [ 12.403036] ixgbe: probe of 0000:03:00.1 failed with error -5 [ 12.408017] initcall ixgbe_init_module+0x0/0x51 returned 0 after 29200 usecs root cause is ioremap can not handle mmio range that is more than 44bits on 32bit PAE mode. We are using pfn with unsigned long like pfn_pte(), so those 0x383fffc00000 will overflow in pfn format with unsigned long (that is 32bits in 32bit x86 kernel, and pfn only can support 44bits). | static inline pte_t pfn_pte(unsigned long page_nr, pgprot_t pgprot) | { | return __pte(((phys_addr_t)page_nr << PAGE_SHIFT) | | massage_pgprot(pgprot)); | } We could limit iomem to 44 bits so we can reject them early from root bus. but xhci is not happy with resource allocation (hang ?) Change phys_addr_t for pfn_pte, and add overflow check to skip ram checking, as the mmio is too big to be ram. At last, can not use PHYSICAL_PAGE_MASK to get aligned phys_addr. Link: https://bugzilla.kernel.org/show_bug.cgi?id=88131 Reported-by: Aaron Ma Tested-by: Aaron Ma Signed-off-by: Yinghai Lu --- arch/x86/include/asm/page.h | 8 ++++++++ arch/x86/include/asm/pgtable.h | 4 ++-- arch/x86/mm/ioremap.c | 6 ++++-- arch/x86/mm/pat.c | 3 +++ 4 files changed, 17 insertions(+), 4 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Index: linux-2.6/arch/x86/include/asm/page.h =================================================================== --- linux-2.6.orig/arch/x86/include/asm/page.h +++ linux-2.6/arch/x86/include/asm/page.h @@ -15,6 +15,14 @@ #ifndef __ASSEMBLY__ +static inline int pfn_overflow(dma_addr_t phy_addr) +{ + dma_addr_t real_pfn = phy_addr >> PAGE_SHIFT; + unsigned long pfn = (unsigned long)real_pfn; + + return pfn != real_pfn; +} + struct page; #include Index: linux-2.6/arch/x86/include/asm/pgtable.h =================================================================== --- linux-2.6.orig/arch/x86/include/asm/pgtable.h +++ linux-2.6/arch/x86/include/asm/pgtable.h @@ -354,9 +354,9 @@ static inline pgprotval_t massage_pgprot return protval; } -static inline pte_t pfn_pte(unsigned long page_nr, pgprot_t pgprot) +static inline pte_t pfn_pte(phys_addr_t page_nr, pgprot_t pgprot) { - return __pte(((phys_addr_t)page_nr << PAGE_SHIFT) | + return __pte((page_nr << PAGE_SHIFT) | massage_pgprot(pgprot)); } Index: linux-2.6/arch/x86/mm/ioremap.c =================================================================== --- linux-2.6.orig/arch/x86/mm/ioremap.c +++ linux-2.6/arch/x86/mm/ioremap.c @@ -122,7 +122,9 @@ static void __iomem *__ioremap_caller(re if (ram_region < 0) { pfn = phys_addr >> PAGE_SHIFT; last_pfn = last_addr >> PAGE_SHIFT; - if (walk_system_ram_range(pfn, last_pfn - pfn + 1, NULL, + /* pfn overflow, don't need to check */ + if (!pfn_overflow(last_addr) && + walk_system_ram_range(pfn, last_pfn - pfn + 1, NULL, __ioremap_check_ram) == 1) return NULL; } @@ -130,7 +132,7 @@ static void __iomem *__ioremap_caller(re * Mappings have to be page-aligned */ offset = phys_addr & ~PAGE_MASK; - phys_addr &= PHYSICAL_PAGE_MASK; + phys_addr -= offset; size = PAGE_ALIGN(last_addr+1) - phys_addr; retval = reserve_memtype(phys_addr, (u64)phys_addr + size, Index: linux-2.6/arch/x86/mm/pat.c =================================================================== --- linux-2.6.orig/arch/x86/mm/pat.c +++ linux-2.6/arch/x86/mm/pat.c @@ -183,6 +183,9 @@ static int pat_pagerange_is_ram(resource unsigned long end_pfn = (end + PAGE_SIZE - 1) >> PAGE_SHIFT; struct pagerange_state state = {start_pfn, 0, 0}; + /* pfn overflow, don't need to check */ + if (pfn_overflow(end + PAGE_SIZE - 1)) + return 0; /* * For legacy reasons, physical address range in the legacy ISA * region is tracked as non-RAM. This will allow users of