From patchwork Wed May 7 23:18:22 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 4132451 X-Patchwork-Delegate: lethal@linux-sh.org Return-Path: X-Original-To: patchwork-linux-sh@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 788DCBFF02 for ; Wed, 7 May 2014 23:18:32 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 6FFBC20222 for ; Wed, 7 May 2014 23:18:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 479F3201C8 for ; Wed, 7 May 2014 23:18:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752892AbaEGXS3 (ORCPT ); Wed, 7 May 2014 19:18:29 -0400 Received: from mail-ie0-f174.google.com ([209.85.223.174]:34223 "EHLO mail-ie0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752890AbaEGXSZ (ORCPT ); Wed, 7 May 2014 19:18:25 -0400 Received: by mail-ie0-f174.google.com with SMTP id ar20so1802653iec.33 for ; Wed, 07 May 2014 16:18:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=a/MsEyUo4Kxu3PsYMqjCLYD2oY5MzJVzTOzSM5q6V20=; b=ojWpmBQ9sX9V2X4Gh5RSuO1DC484aAf3SlFjRCf5n1P8NdeQKMkdj8Iq+ksav7U7Io 1uX3nFPCPzOyKv6X0qg7lr2OLNA1Zh72JRoChtGBqT2bFMGTPFdhBfn4B6+WNEsg3l9n sSovbGRK5XuL4CkUJv9sEboILSgMiQO3Ycw2uqLrQuIsulUJXq0dz0mZjuefrwwutFE2 bYPE6RG3NHDO5QcfmnqfaRW3E84hbnFRbFpIcs0rjpVGu/gwgbe8sPoZp8Qcg+3a+3lU dpXglc8fhq0J9tZ7g2nKFz5oYhJUvzRTxvYaGJEBJEWb28QOMEHzUz0KelrVEVemBhEp 4gNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=a/MsEyUo4Kxu3PsYMqjCLYD2oY5MzJVzTOzSM5q6V20=; b=N+f9BLpbQ3GK3c0JCjgRIF4nSmlqZpt0DPgV7NnGFn+Ez8pyev4BvtX1FAgooxmqiW 7DZBIRl8I6n8Y6MBcfM/AW6WTwyqysvus/dywltHQLouzLu8Q6BA31WCminl7FsiXPGn Sr7M1v5FGSuA123RfzBtFEmJH4dWFhUy+LEtacoZ/8CFX3DBZUbokUuhRimxuGTdKJwy US0a8OhDcEI2qBqBV5wx8IDtkLSibzgcZ+CX6M79SC9RoNx45Hi+cyu/3teEM63gB5db J7veW4DDKixisGhdK++q1QqLm1w6nUHfNBtYjPcuWej9BiseVeQ9e//TUD9tjJSvaGsx TLYg== X-Gm-Message-State: ALoCoQkVN7s9LY32FEid0unCyH7XgABl68CGT/Ce/8cfBN0eP6cHITBKWwW9sfJw1EvGzqk2HtYa X-Received: by 10.51.15.161 with SMTP id fp1mr1571272igd.25.1399504705167; Wed, 07 May 2014 16:18:25 -0700 (PDT) Received: from google.com ([172.16.48.245]) by mx.google.com with ESMTPSA id 2sm1023308igz.20.2014.05.07.16.18.24 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Wed, 07 May 2014 16:18:24 -0700 (PDT) Date: Wed, 7 May 2014 17:18:22 -0600 From: Bjorn Helgaas To: Arnd Bergmann Cc: linux-doc@vger.kernel.org, Greg Kroah-Hartman , Joerg Roedel , Randy Dunlap , Liviu Dudau , linux-kernel@vger.kernel.org, James Bottomley , linux-pci@vger.kernel.org, David Woodhouse , Magnus Damm , linux-sh@vger.kernel.org Subject: Re: [PATCH v2 3/5] sh/PCI: Pass GAPSPCI_DMA_BASE CPU address to dma_declare_coherent_memory() Message-ID: <20140507231822.GA4951@google.com> References: <20140506223250.17968.27054.stgit@bhelgaas-glaptop.roam.corp.google.com> <20140506224833.17968.87108.stgit@bhelgaas-glaptop.roam.corp.google.com> <6994558.Q3OYh8PWMa@wuerfel> <4271645.KYLAoYxC5v@wuerfel> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <4271645.KYLAoYxC5v@wuerfel> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-sh-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sh@vger.kernel.org X-Spam-Status: No, score=-7.4 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP [+cc Magnus, linux-sh (sorry, I forgot to tell stgit to cc you)] On Wed, May 07, 2014 at 10:15:10AM +0200, Arnd Bergmann wrote: > On Wednesday 07 May 2014 09:55:16 Arnd Bergmann wrote: > > On Tuesday 06 May 2014 16:48:33 Bjorn Helgaas wrote: > > > @@ -51,8 +53,12 @@ static void gapspci_fixup_resources(struct pci_dev *dev) > > > /* > > > * Redirect dma memory allocations to special memory window. > > > */ > > > + region.start = GAPSPCI_DMA_BASE; > > > + region.end = GAPSPCI_DMA_BASE + GAPSPCI_DMA_SIZE - 1; > > > + res.flags = IORESOURCE_MEM; > > > + pcibios_bus_to_resource(dev->bus, &res, ®ion); > > > BUG_ON(!dma_declare_coherent_memory(&dev->dev, > > > - GAPSPCI_DMA_BASE, > > > + res->start, > > > GAPSPCI_DMA_BASE, > > > GAPSPCI_DMA_SIZE, > > > DMA_MEMORY_MAP | > > > > Not sure if this is an improvement. I understand the intention, but > > it's actually possible for the offset to be different both ways: > > > > Your patch applies the outbound mem_offset that was provided when > > registering the MMIO resource for the PCI host bridge. What the driver > > needs instead is the inbound DMA offset that gets applied by some > > host bridges that don't have a 1:1 mapping but also don't have an > > IOMMU. I don't understand where the inbound DMA offset comes into play. As I understand it, we're talking about a region of memory on the PCI bus, not in system RAM, and there are two ways to access it: (1) the CPU makes MMIO accesses to it, and (2) the device makes DMA accesses to it. The CPU accesses go through the host bridge MMIO aperture and will have the outbound mem_offset applied to them. The DMA accesses are handled by the device itself and there's no host bridge or IOMMU or inbound DMA offset involved. I think there are two reasonable ways a PCI driver could use dma_declare_coherent_memory(): 1) The bus address of the region might be in a BAR. In that case, it should pass pci_resource_start() (the CPU phys_addr_t) as the first address, and pci_bus_address(pci_resource_start()) (the bus dma_addr_t) as the second. 2) The bus address of the region might be discovered from the device in some non-standard way. In this case, the driver reads the bus address dma_addr_t directly from the device, and it should use something like the pcibios_bus_to_resource() code I proposed to find the CPU phys_addr_t. For this platform, all the addresses are hard-coded and there is no outbound MMIO offset, so we can't tell which to use and in this system, it doesn't matter if we do anything at all. The only reason I would make a change here is because the current code cannot be safely copied (another driver might be used in systems where the host bridge does have an outbound MMIO offset). If we make this code use either strategy, it would be a clue to future users that they should not assume the physical address is the same as the bus address. I changed the patch (below) to add a comment and to use the first strategy (though there isn't an actual BAR, so we can't do it exactly that way). > > We know that on this particular platform, both are zero, so > > the original code works and it will keep working with your change, > > but I think it's a mistake anyway. I have seen both kinds of offsets > > in the past on real machines, but I am not aware of any where they > > are both nonzero and have the same value. > > Hmm, looking at it again, it seem even weirder: the address we register > for DMA allocations is the same that gets passed into the PCI host > for MMIO resources. Something is very strange here. Still, I'd rather > not touch it at all, whatever it does. It's definitely strange. It looks to me like the memory on the device takes up the entire 32KB host bridge MMIO aperture. I don't know what driver uses this, but the device must be programmed via I/O ports, which sort of makes sense given the previous lines in this quirk that set the BAR1 resource to be in the I/O aperture. Bjorn sh/PCI: Pass GAPSPCI_DMA_BASE CPU & bus address to dma_declare_coherent_memory() From: Bjorn Helgaas dma_declare_coherent_memory() needs both the CPU physical address and the bus address of the device memory. They are the same on this platform, but in general we should use pcibios_resource_to_bus() to account for any address translation done by the PCI host bridge. This makes no difference on Dreamcast, but is safer if the usage is copied to future drivers. Signed-off-by: Bjorn Helgaas CC: Magnus Damm CC: linux-sh@vger.kernel.org Acked-by: Arnd Bergmann --- arch/sh/drivers/pci/fixups-dreamcast.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-sh" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/arch/sh/drivers/pci/fixups-dreamcast.c b/arch/sh/drivers/pci/fixups-dreamcast.c index d6cde700e316..f22127f966ce 100644 --- a/arch/sh/drivers/pci/fixups-dreamcast.c +++ b/arch/sh/drivers/pci/fixups-dreamcast.c @@ -31,6 +31,8 @@ static void gapspci_fixup_resources(struct pci_dev *dev) { struct pci_channel *p = dev->sysdata; + struct resource res; + struct pci_bus_region region; printk(KERN_NOTICE "PCI: Fixing up device %s\n", pci_name(dev)); @@ -50,11 +52,21 @@ static void gapspci_fixup_resources(struct pci_dev *dev) /* * Redirect dma memory allocations to special memory window. + * + * If this GAPSPCI region were mapped by a BAR, the CPU + * phys_addr_t would be pci_resource_start(), and the bus + * address would be pci_bus_address(pci_resource_start()). + * But apparently there's no BAR mapping it, so we just + * "know" its CPU address is GAPSPCI_DMA_BASE. */ + res.start = GAPSPCI_DMA_BASE; + res.end = GAPSPCI_DMA_BASE + GAPSPCI_DMA_SIZE - 1; + res.flags = IORESOURCE_MEM; + pcibios_resource_to_bus(dev, ®ion, &resource); BUG_ON(!dma_declare_coherent_memory(&dev->dev, - GAPSPCI_DMA_BASE, - GAPSPCI_DMA_BASE, - GAPSPCI_DMA_SIZE, + res.start, + region.start, + resource_size(&res), DMA_MEMORY_MAP | DMA_MEMORY_EXCLUSIVE)); break;