From patchwork Tue Apr 30 11:00:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 10923249 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3CBBF17DF for ; Tue, 30 Apr 2019 11:01:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2E25328A1E for ; Tue, 30 Apr 2019 11:01:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2274328A3A; Tue, 30 Apr 2019 11:01:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AAF5D28A24 for ; Tue, 30 Apr 2019 11:01:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727639AbfD3LBW (ORCPT ); Tue, 30 Apr 2019 07:01:22 -0400 Received: from bombadil.infradead.org ([198.137.202.133]:43238 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727613AbfD3LBS (ORCPT ); Tue, 30 Apr 2019 07:01:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From :Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=hxewYvCmTQdqR9W0kvD+1bajCF53IT9u1fR91BrTg28=; b=anCeBpI+Elx7eUtxG6lKG2HsI4 xhZXkRxzZyuklPGUdThFNgMknmV0dGsjF+m5hjiMdinCFRxlWr4CNCBPZ4hraY/OcGgHcfMU82rGy X9UUp01qHzL9wcav1oMFe6LfwKNZk/1mRtJyo4bsLfIu/Lcnypsb37AFyEGpl+BJf9sFmvkP7pIEn lnVfmHI/YoMhFrDlNkAgrdm+zsNsJWOFwit7AO3TS3hjSO7x0bpvhHNod8/WDlwERvEPg112RzwZ3 9SQcNPBCVKvSDrDalwnRU5M/7h1Gt+RRoYxY35ioJ+FJaaO7F4RX7QWn9C2z4B/+XUfJxeq8b85BT UnLXrKVw==; Received: from adsl-173-228-226-134.prtc.net ([173.228.226.134] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.90_1 #2 (Red Hat Linux)) id 1hLQVg-0000HF-Md; Tue, 30 Apr 2019 11:01:16 +0000 From: Christoph Hellwig To: Ralf Baechle , Paul Burton , James Hogan , Ley Foon Tan , Michal Simek , Bartlomiej Zolnierkiewicz Cc: linux-mips@vger.kernel.org, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, linux-fbdev@vger.kernel.org Subject: [PATCH 4/7] dma-direct: provide generic support for uncached kernel segments Date: Tue, 30 Apr 2019 07:00:29 -0400 Message-Id: <20190430110032.25301-5-hch@lst.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190430110032.25301-1-hch@lst.de> References: <20190430110032.25301-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Sender: linux-fbdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fbdev@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP A few architectures support uncached kernel segments. In that case we get an uncached mapping for a given physica address by using an offset in the uncached segement. Implement support for this scheme in the generic dma-direct code instead of duplicating it in arch hooks. Signed-off-by: Christoph Hellwig Signed-off-by: Christoph Hellwig Signed-off-by: Christoph Hellwig --- arch/Kconfig | 8 ++++++++ include/linux/dma-noncoherent.h | 3 +++ kernel/dma/direct.c | 18 ++++++++++++++++-- 3 files changed, 27 insertions(+), 2 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 33687dddd86a..ea22a8c894ec 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -249,6 +249,14 @@ config ARCH_HAS_FORTIFY_SOURCE config ARCH_HAS_SET_MEMORY bool +# +# Select if arch has an uncached kernel segment and provides the +# uncached_kernel_address / cached_kernel_address symbols to use it +# +config ARCH_HAS_UNCACHED_SEGMENT + select ARCH_HAS_DMA_PREP_COHERENT + bool + # Select if arch init_task must go in the __init_task_data section config ARCH_TASK_STRUCT_ON_STACK bool diff --git a/include/linux/dma-noncoherent.h b/include/linux/dma-noncoherent.h index 9741767e400f..7e0126a04e02 100644 --- a/include/linux/dma-noncoherent.h +++ b/include/linux/dma-noncoherent.h @@ -80,4 +80,7 @@ static inline void arch_dma_prep_coherent(struct page *page, size_t size) } #endif /* CONFIG_ARCH_HAS_DMA_PREP_COHERENT */ +void *uncached_kernel_address(void *addr); +void *cached_kernel_address(void *addr); + #endif /* _LINUX_DMA_NONCOHERENT_H */ diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index 2c2772e9702a..d15a535c3e67 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -164,6 +164,13 @@ void *dma_direct_alloc_pages(struct device *dev, size_t size, } ret = page_address(page); + + if (IS_ENABLED(CONFIG_ARCH_HAS_UNCACHED_SEGMENT) && + !dev_is_dma_coherent(dev) && !(attrs & DMA_ATTR_NON_CONSISTENT)) { + arch_dma_prep_coherent(page, size); + ret = uncached_kernel_address(ret); + } + if (force_dma_unencrypted()) { set_memory_decrypted((unsigned long)ret, 1 << get_order(size)); *dma_handle = __phys_to_dma(dev, page_to_phys(page)); @@ -171,6 +178,7 @@ void *dma_direct_alloc_pages(struct device *dev, size_t size, *dma_handle = phys_to_dma(dev, page_to_phys(page)); } memset(ret, 0, size); + return ret; } @@ -189,13 +197,18 @@ void dma_direct_free_pages(struct device *dev, size_t size, void *cpu_addr, if (force_dma_unencrypted()) set_memory_encrypted((unsigned long)cpu_addr, 1 << page_order); + + if (IS_ENABLED(CONFIG_ARCH_HAS_UNCACHED_SEGMENT) && + !dev_is_dma_coherent(dev) && !(attrs & DMA_ATTR_NON_CONSISTENT)) + cpu_addr = cached_kernel_address(cpu_addr); __dma_direct_free_pages(dev, size, virt_to_page(cpu_addr)); } void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs) { - if (!dev_is_dma_coherent(dev)) + if (!IS_ENABLED(CONFIG_ARCH_HAS_UNCACHED_SEGMENT) && + !dev_is_dma_coherent(dev)) return arch_dma_alloc(dev, size, dma_handle, gfp, attrs); return dma_direct_alloc_pages(dev, size, dma_handle, gfp, attrs); } @@ -203,7 +216,8 @@ void *dma_direct_alloc(struct device *dev, size_t size, void dma_direct_free(struct device *dev, size_t size, void *cpu_addr, dma_addr_t dma_addr, unsigned long attrs) { - if (!dev_is_dma_coherent(dev)) + if (!IS_ENABLED(CONFIG_ARCH_HAS_UNCACHED_SEGMENT) && + !dev_is_dma_coherent(dev)) arch_dma_free(dev, size, cpu_addr, dma_addr, attrs); else dma_direct_free_pages(dev, size, cpu_addr, dma_addr, attrs);