From patchwork Mon Sep 9 16:46:59 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Stabellini X-Patchwork-Id: 2862571 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 4FFB29F495 for ; Mon, 9 Sep 2013 16:52:06 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 1CFC520379 for ; Mon, 9 Sep 2013 16:52:05 +0000 (UTC) Received: from casper.infradead.org (casper.infradead.org [85.118.1.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4BDB92024A for ; Mon, 9 Sep 2013 16:52:00 +0000 (UTC) Received: from merlin.infradead.org ([2001:4978:20e::2]) by casper.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1VJ4gv-0006tC-M1; Mon, 09 Sep 2013 16:51:57 +0000 Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1VJ4gt-00042a-8u; Mon, 09 Sep 2013 16:51:55 +0000 Received: from smtp.citrix.com ([66.165.176.89]) by merlin.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1VJ4go-00041p-Qt for linux-arm-kernel@lists.infradead.org; Mon, 09 Sep 2013 16:51:53 +0000 X-IronPort-AV: E=Sophos;i="4.90,872,1371081600"; d="scan'208";a="52062611" Received: from accessns.citrite.net (HELO FTLPEX01CL03.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 09 Sep 2013 16:51:26 +0000 Received: from ukmail1.uk.xensource.com (10.80.16.128) by smtprelay.citrix.com (10.13.107.80) with Microsoft SMTP Server id 14.2.342.4; Mon, 9 Sep 2013 12:51:26 -0400 Received: from kaball.uk.xensource.com ([10.80.2.59]) by ukmail1.uk.xensource.com with esmtp (Exim 4.69) (envelope-from ) id 1VJ4cT-0006Gg-ET; Mon, 09 Sep 2013 17:47:21 +0100 Date: Mon, 9 Sep 2013 17:46:59 +0100 From: Stefano Stabellini X-X-Sender: sstabellini@kaball.uk.xensource.com To: Catalin Marinas Subject: Re: [PATCH v5 11/13] xen: introduce xen_alloc/free_coherent_pages In-Reply-To: <8492887B-2509-42B5-A5F9-A711C519D912@arm.com> Message-ID: References: <1377801154-29215-11-git-send-email-stefano.stabellini@eu.citrix.com> <20130905160905.GI11887@arm.com> <20130906141444.GE1946@arm.com> <20130906155928.GH1946@arm.com> <20130906162018.GJ1946@arm.com> <8492887B-2509-42B5-A5F9-A711C519D912@arm.com> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 X-DLP: MIA1 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20130909_125151_103838_C42FA522 X-CRM114-Status: GOOD ( 22.81 ) X-Spam-Score: -7.8 (-------) Cc: "xen-devel@lists.xensource.com" , "Ian.Campbell@citrix.com" , "konrad.wilk@oracle.com" , Stefano Stabellini , "linux-kernel@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-5.1 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Mon, 9 Sep 2013, Catalin Marinas wrote: > >>> They could also happen in a DomU if we assign a physical device to it > >>> (and an SMMU is not available). > >> > >> The problem is that you don't necessarily know one kind of coherency you > >> know for a physical device. As I said, we plan to do this DT-driven. > > > > OK, but if I call arm64_swiotlb_dma_ops.alloc passing the right > > arguments to it, I should be able to get the right coherency for the > > right device, correct? > > I think it needs a bit more work on the Xen part. Basically > dma_alloc_attrs() calls get_dma_ops() to obtain the best DMA operations > for a device. arm64_swiotlb_dma_ops is just the default implementation > and I'll add a _noncoherent variant as well. Default dma_ops will be > set to one of these during boot. But a device is also allowed to have > its own dev->archdata.dma_ops, set via set_dma_ops(). > > So even if you set the default dma_ops to Xen ops, you may not get them > via dma_alloc_coherent(). I don't see any easier solution other than > patching the dma_alloc_attrs() function to issue a Hyp call after the > memory has been allocated with the get_dma_ops()->alloc(). But I don't > like this either. I see. This problem affects arch/arm as well. Either we add an if (!xen_domain()) in get_dma_ops, or we could make get_dma_ops a function pointer and let people overwrite it. See below the first option implemented for arch/arm on top of the swiotlb series: diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h index 7d6e4f9..0b8b5e4 100644 --- a/arch/arm/include/asm/dma-mapping.h +++ b/arch/arm/include/asm/dma-mapping.h @@ -12,6 +12,8 @@ #include #include +#include + #define DMA_ERROR_CODE (~0) extern struct dma_map_ops *dma_ops; extern struct dma_map_ops arm_dma_ops; @@ -19,7 +21,7 @@ extern struct dma_map_ops arm_coherent_dma_ops; static inline struct dma_map_ops *get_dma_ops(struct device *dev) { - if (dev && dev->archdata.dma_ops) + if (!xen_domain() && dev && dev->archdata.dma_ops) return dev->archdata.dma_ops; return dma_ops; } diff --git a/arch/arm/include/asm/xen/page-coherent.h b/arch/arm/include/asm/xen/page-coherent.h index af2cf8d..c2232fe 100644 --- a/arch/arm/include/asm/xen/page-coherent.h +++ b/arch/arm/include/asm/xen/page-coherent.h @@ -9,6 +9,8 @@ static inline void *xen_alloc_coherent_pages(struct device *hwdev, size_t size, dma_addr_t *dma_handle, gfp_t flags, struct dma_attrs *attrs) { + if (hwdev && hwdev->archdata.dma_ops) + return hwdev->archdata.dma_ops->alloc(hwdev, size, dma_handle, flags, attrs); return arm_dma_ops.alloc(hwdev, size, dma_handle, flags, attrs); } @@ -16,6 +18,8 @@ static inline void xen_free_coherent_pages(struct device *hwdev, size_t size, void *cpu_addr, dma_addr_t dma_handle, struct dma_attrs *attrs) { + if (hwdev && hwdev->archdata.dma_ops) + return hwdev->archdata.dma_ops->free(hwdev, size, cpu_addr, dma_handle, attrs); return arm_dma_ops.free(hwdev, size, cpu_addr, dma_handle, attrs); }