From patchwork Mon Mar 17 23:52:40 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 3851711 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id DD2859F3FF for ; Wed, 19 Mar 2014 17:33:47 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id D8C67203AF for ; Wed, 19 Mar 2014 17:33:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B3FF220381 for ; Wed, 19 Mar 2014 17:33:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751292AbaCQXwr (ORCPT ); Mon, 17 Mar 2014 19:52:47 -0400 Received: from mail-pa0-f43.google.com ([209.85.220.43]:40699 "EHLO mail-pa0-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751200AbaCQXwp (ORCPT ); Mon, 17 Mar 2014 19:52:45 -0400 Received: by mail-pa0-f43.google.com with SMTP id bj1so6429342pad.30 for ; Mon, 17 Mar 2014 16:52:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=/2XQNLbgfhXtwaxXr2HjDddl/J/jiLY8RUiT7cPKWDU=; b=GTdaWNVE4xeLHsUHNdn/qfTw4IQZm0oKWwXPWLd0f8jKjITY1g9ACtnuquUR2Awb4a 9pL7NMZ70G5Pr3w7C5Lb7o9CZh6k0gz0qSWkQjK5u6J06nj1qlTel497vyQdwhdAj6hY jXtHuGTBx/QfDAjPRm3KlQHcJtNw34xrYoaPLA9S59BSqgovMI7Q0IqgtP3GamVCR0fm Ylsw3zvpOZ6lCEdPDO3cVchQxp8rcHfYcZwxZAjKI7EO65OOQH8y5ht+0990cU1HyCU4 4ic8FZ0TutUWQ5JnbTi/DbSVzpyMe7Wi4z2QuN3FIZ/xNnlTkiWhUKcZLDFdalyiAhCy LiCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-type:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=/2XQNLbgfhXtwaxXr2HjDddl/J/jiLY8RUiT7cPKWDU=; b=WS9g3jIfpMP7Z3J9q3oIyArpeAgx1d23kullD81AE8X35XL3sAtnqMhLg7Txo97XiX AxmOpMr9JU43FOI8iiV5oJx8ZMvqCOrZ8VBlyECi/JNbSb2WUj+52GZhfr1erDJteD4z ad6O/9mWXeiFZnfEoIueIaCbPE1wjlSAXuCIp9uojS7wzjiqbUd44eur/Cy5+/u/Rh6i kIPHsXuz69TGnrod+RRV4UTCpjZl5wdZw6mlblTRgGIy/7xIXr7VrdHbh25PPkyoYqle Rbps/Ih42atqh7XtQPaYkFZOeMTqtYvkPrLl2uiMYgzAKaFz9GHaodL2OlRclDax0Mqd mqnw== X-Gm-Message-State: ALoCoQm19Uem6WwqdHhF0dbLfDmtSv3+luq7lDmcyC/LU/fgR01DlwtkNFG2dnhU4NqFhNI4gz/5ZItcDCeX0kWrk4HPYXOQpJEDNpBPnZJ+mFqItups9pAJG2FRM3HzAx8oyD/mhViimq2CpVzk/glDjvwJ1GTxJFq8tuEPpcHuyxk2ERJLbttbN7mHYM4dYbTL3hbO5vq9yODtfzDIXt4+5J/+ULCnww== X-Received: by 10.67.2.34 with SMTP id bl2mr29067337pad.58.1395100365121; Mon, 17 Mar 2014 16:52:45 -0700 (PDT) Received: from google.com ([172.29.164.218]) by mx.google.com with ESMTPSA id gj9sm46801672pbc.7.2014.03.17.16.52.43 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 17 Mar 2014 16:52:44 -0700 (PDT) Date: Mon, 17 Mar 2014 17:52:40 -0600 From: Bjorn Helgaas To: Jouni =?iso-8859-1?Q?Mett=E4l=E4?= Cc: linux-pci@vger.kernel.org, Daniel Vetter , Yinghai Lu , Guo Chao , David Airlie , Aaron Durbin , linux-kernel@vger.kernel.org Subject: Re: [bisected] e501 "agp: Support 64-bit APBASE" agp fails without iommu=remap=2 Message-ID: <20140317235240.GA14292@google.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,RCVD_IN_DNSWL_HI,T_DKIM_INVALID,T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP [+cc Daniel, Dave, Yinghai, Guo, Aaron, LKML] On Sat, Mar 15, 2014 at 02:32:22PM +0200, Jouni Mettälä wrote: > Hi. I can't start xsession without iommu=remap=2 kernel parameter. > > Kernel bisect lead to this commit > http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=e501b3d87f003dfad8fcbd0f55ae17ea52495a56 > > Reverting e501b3d87f003dfad8fcbd0f55ae17ea52495a56 fixes if for me. > > I think this is relevant part of dmesg from affected kernel > > agpgart-amd64 0000:00:04.0: AGP bridge [10b9/1689] > agpgart-amd64 0000:00:04.0: aperture from AGP @ 0 size 4096 MB > pci 0000:00:18.3: no usable aperture found > pci 0000:00:18.3: consider rebooting with iommu=memaper=2 to get a good > aperture > agpgart-amd64 0000:00:04.0: AGP bridge [10b9/1689] > agpgart-amd64 0000:00:04.0: aperture from AGP @ 0 size 4096 MB > pci 0000:00:18.3: no usable aperture found > pci 0000:00:18.3: consider rebooting with iommu=memaper=2 to get a good > aperture > agpgart-amd64 0000:04:00.0: AGP bridge [1002/4150] > agpgart-amd64 0000:04:00.0: aperture size 4096 MB is not right, using > settings from NB > agpgart-amd64 0000:04:00.0: AGP aperture is 256M @ 0xa0000000 Jouni also opened this bug report with complete dmesg logs: https://bugzilla.kernel.org/show_bug.cgi?id=72201 This is a regression that appeared in v3.14-rc1, when we merged e501b3d87f00. The relevant part of that change is this: --- a/drivers/char/agp/amd64-agp.c +++ b/drivers/char/agp/amd64-agp.c @@ -295,9 +294,7 @@ static int fix_northbridge(struct pci_dev *nb, struct pci_dev *agp, u16 cap) - pci_read_config_dword(agp, 0x10, &aper_low); - pci_read_config_dword(agp, 0x14, &aper_hi); - aper = (aper_low & ~((1<<22)-1)) | ((u64)aper_hi << 32); + aper = pci_bus_address(agp, AGP_APERTURE_BAR); Here's what I think is happening: Previously, we read the GART aperture base directly from the BAR. After e501b3d87f00, we use pci_bus_address() to convert the aperture *resource* (which the PCI core has previously read from the BAR and converted to a CPU address) back into a bus address. Normally both ways would give the same result, but here we had this: Node 0: aperture @ a0000000 size 256 MB pci 0000:00:04.0: reg 0x10: [mem 0xa0000000-0xafffffff pref] pci 0000:00:04.0: address space collision: [mem 0xa0000000-0xafffffff pref] conflicts with GART [mem 0xa0000000-0xafffffff] The "Node 0" line is where we inserted the "GART [mem 0xa0000000- 0xafffffff]" resource in gart_iommu_hole_init(). Then we enumerated the northbridge, which had a BAR containing 0xa0000000. When we tried to claim that BAR, it conflicted with the "GART" resource, and we set r->start = 0 (in pcibios_allocate_dev_resources()). So when we finally got to fix_northbridge(), the aperture resource was set to zero, not 0xa0000000. Note that we complained about the collision even before e501b3d87f00. The only difference is that we used to re-read the BAR, where we still got 0xa0000000 even though the PCI core thought the resource was invalid and had set it to zero. I think we should stop inserting the GART resource directly in iomem_resource in gart_iommu_hole_init(). That should avoid the collision and leave the BAR resource valid, which means pci_bus_address() should work. Jouni, could you try the patch below? Bjorn Revert "[PATCH] Insert GART region into resource map" From: Bjorn Helgaas This reverts commit 56dd669a138c, which makes the GART visible in /proc/iomem. The GART addresses are bus addresses, not CPU addresses, and therefore should not be inserted in iomem_resource. On many machines, the GART region is addressable by the CPU as well as by an AGP master, but CPU addressability is not required by the spec. On some of these machines, the GART is mapped by a PCI BAR, and in that case, the PCI core automatically inserts it into iomem_resource, just as it does for all BARs. Inserting it here means we'll have a conflict if the PCI core later tries to claim the GART region. Conflicts: arch/x86_64/kernel/aperture.c Signed-off-by: Bjorn Helgaas --- arch/x86/kernel/aperture_64.c | 20 +------------------- 1 file changed, 1 insertion(+), 19 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/arch/x86/kernel/aperture_64.c b/arch/x86/kernel/aperture_64.c index fd972a3e4cbb..9fa8aa051f54 100644 --- a/arch/x86/kernel/aperture_64.c +++ b/arch/x86/kernel/aperture_64.c @@ -18,7 +18,6 @@ #include #include #include -#include #include #include #include @@ -54,18 +53,6 @@ int fallback_aper_force __initdata; int fix_aperture __initdata = 1; -static struct resource gart_resource = { - .name = "GART", - .flags = IORESOURCE_MEM, -}; - -static void __init insert_aperture_resource(u32 aper_base, u32 aper_size) -{ - gart_resource.start = aper_base; - gart_resource.end = aper_base + aper_size - 1; - insert_resource(&iomem_resource, &gart_resource); -} - /* This code runs before the PCI subsystem is initialized, so just access the northbridge directly. */ @@ -96,7 +83,6 @@ static u32 __init allocate_aperture(void) memblock_reserve(addr, aper_size); printk(KERN_INFO "Mapping aperture over %d KB of RAM @ %lx\n", aper_size >> 10, addr); - insert_aperture_resource((u32)addr, aper_size); register_nosave_region(addr >> PAGE_SHIFT, (addr+aper_size) >> PAGE_SHIFT); @@ -444,12 +430,8 @@ int __init gart_iommu_hole_init(void) out: if (!fix && !fallback_aper_force) { - if (last_aper_base) { - unsigned long n = (32 * 1024 * 1024) << last_aper_order; - - insert_aperture_resource((u32)last_aper_base, n); + if (last_aper_base) return 1; - } return 0; }