From patchwork Tue Jul 3 10:37:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Angelo Dureghello X-Patchwork-Id: 10503773 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D9CFB601D3 for ; Tue, 3 Jul 2018 10:37:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C34A2287E5 for ; Tue, 3 Jul 2018 10:37:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B73E0289AD; Tue, 3 Jul 2018 10:37:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F1BA0287E5 for ; Tue, 3 Jul 2018 10:37:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754728AbeGCKhZ (ORCPT ); Tue, 3 Jul 2018 06:37:25 -0400 Received: from ec2-18-194-220-216.eu-central-1.compute.amazonaws.com ([18.194.220.216]:59694 "EHLO sysam.it" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754665AbeGCKhX (ORCPT ); Tue, 3 Jul 2018 06:37:23 -0400 Received: from localhost (localhost [127.0.0.1]) by sysam.it (Postfix) with ESMTP id 3208221B01; Tue, 3 Jul 2018 10:37:22 +0000 (UTC) Received: from sysam.it ([127.0.0.1]) by localhost (sysam.it [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2oPbVXa4pIN7; Tue, 3 Jul 2018 10:37:20 +0000 (UTC) Received: from localhost.localdomain (host51-24-dynamic.54-79-r.retail.telecomitalia.it [79.54.24.51]) by sysam.it (Postfix) with ESMTPSA id CD1CA21AFE; Tue, 3 Jul 2018 10:37:18 +0000 (UTC) From: Angelo Dureghello To: vinod.koul@linaro.org Cc: dmaengine@vger.kernel.org, linux-m68k@vger.kernel.org, Angelo Dureghello Subject: [PATCH v7 1/2] dmaengine: fsl-edma: extract common fsl-edma code (no changes in behavior intended) Date: Tue, 3 Jul 2018 12:37:15 +0200 Message-Id: <20180703103716.20482-1-angelo@sysam.it> X-Mailer: git-send-email 2.18.0 Sender: dmaengine-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch adds a new fsl-edma-common module to allow new mcf-edma module code to use most of the fsl-edma code. Due to some differences between ColdFire edma (64 channels) and fsl-edma (32 channels), as register set offsets and some other points as the different interrupt organization and other minor things, a common module can collect most of the code for both 32 and 64 channel edma module versions. Signed-off-by: Angelo Dureghello --- Changes for v2: - patch splitted into 4 - add fsl-edma-common module Changes for v3: none Changes for v4: - patch simplified from 4/4 into 2/2 - collecting all the fsl-edma-related changes Changes for v5: - add EXPORT_SYMBOL_GPL for all common functions Changes for v6: - adjusted comment header - we need to free the interrupts at remove(), so removed all devm_ interrupt related calls Changes for v7: - fix sparse errors - fix additional space in header file end --- drivers/dma/Makefile | 2 +- drivers/dma/fsl-edma-common.c | 670 ++++++++++++++++++++++++++++++ drivers/dma/fsl-edma-common.h | 161 ++++++++ drivers/dma/fsl-edma.c | 753 ++-------------------------------- 4 files changed, 870 insertions(+), 716 deletions(-) create mode 100644 drivers/dma/fsl-edma-common.c create mode 100644 drivers/dma/fsl-edma-common.h diff --git a/drivers/dma/Makefile b/drivers/dma/Makefile index 203a99d68315..66022f59fca4 100644 --- a/drivers/dma/Makefile +++ b/drivers/dma/Makefile @@ -31,7 +31,7 @@ obj-$(CONFIG_DW_AXI_DMAC) += dw-axi-dmac/ obj-$(CONFIG_DW_DMAC_CORE) += dw/ obj-$(CONFIG_EP93XX_DMA) += ep93xx_dma.o obj-$(CONFIG_FSL_DMA) += fsldma.o -obj-$(CONFIG_FSL_EDMA) += fsl-edma.o +obj-$(CONFIG_FSL_EDMA) += fsl-edma.o fsl-edma-common.o obj-$(CONFIG_FSL_RAID) += fsl_raid.o obj-$(CONFIG_HSU_DMA) += hsu/ obj-$(CONFIG_IMG_MDC_DMA) += img-mdc-dma.o diff --git a/drivers/dma/fsl-edma-common.c b/drivers/dma/fsl-edma-common.c new file mode 100644 index 000000000000..72e52c1b5527 --- /dev/null +++ b/drivers/dma/fsl-edma-common.c @@ -0,0 +1,670 @@ +// SPDX-License-Identifier: GPL-2.0 +// +// Copyright (c) 2013-2014 Freescale Semiconductor, Inc +// Copyright (c) 2017 Sysam, Angelo Dureghello + +#include +#include + +#include "fsl-edma-common.h" + +#define EDMA_CR 0x00 +#define EDMA_ES 0x04 +#define EDMA_ERQ 0x0C +#define EDMA_EEI 0x14 +#define EDMA_CEEI 0x18 +#define EDMA_SEEI 0x19 +#define EDMA_CERQ 0x1A +#define EDMA_SERQ 0x1B +#define EDMA_CINT 0x1F +#define EDMA_CERR 0x1E +#define EDMA_SSRT 0x1D +#define EDMA_CDNE 0x1C +#define EDMA_INTR 0x24 +#define EDMA_ERR 0x2C + +#define EDMA64_ERQH 0x08 +#define EDMA64_EEIH 0x10 +#define EDMA64_SERQ 0x18 +#define EDMA64_CERQ 0x19 +#define EDMA64_SEEI 0x1a +#define EDMA64_CEEI 0x1b +#define EDMA64_CINT 0x1c +#define EDMA64_CERR 0x1d +#define EDMA64_SSRT 0x1e +#define EDMA64_CDNE 0x1f +#define EDMA64_INTH 0x20 +#define EDMA64_INTL 0x24 +#define EDMA64_ERRH 0x28 +#define EDMA64_ERRL 0x2c + +#define EDMA_TCD 0x1000 +#define EDMA_TCD_SIZE 32 +#define EDMA_TCD_MEM_ALIGN EDMA_TCD_SIZE + +#define EDMA_TCD_ATTR_SSIZE_8BIT 0 +#define EDMA_TCD_ATTR_SSIZE_16BIT BIT(0) +#define EDMA_TCD_ATTR_SSIZE_32BIT BIT(1) +#define EDMA_TCD_ATTR_SSIZE_64BIT (BIT(1) | BIT(0)) +#define EDMA_TCD_ATTR_SSIZE_16BYTE BIT(2) +#define EDMA_TCD_ATTR_SSIZE_32BYTE (BIT(2) | BIT(0)) +#define EDMA_TCD_ATTR_DSIZE_8BIT (EDMA_TCD_ATTR_SSIZE_8BIT << 8) +#define EDMA_TCD_ATTR_DSIZE_16BIT (EDMA_TCD_ATTR_SSIZE_16BIT << 8) +#define EDMA_TCD_ATTR_DSIZE_32BIT (EDMA_TCD_ATTR_SSIZE_32BIT << 8) +#define EDMA_TCD_ATTR_DSIZE_64BIT (EDMA_TCD_ATTR_SSIZE_64BIT << 8) +#define EDMA_TCD_ATTR_DSIZE_16BYTE (EDMA_TCD_ATTR_SSIZE_16BYTE << 8) +#define EDMA_TCD_ATTR_DSIZE_32BYTE (EDMA_TCD_ATTR_SSIZE_32BYTE << 8) + +#define EDMA_TCD_CITER_CITER(x) ((x) & GENMASK(14, 0)) +#define EDMA_TCD_BITER_BITER(x) ((x) & GENMASK(14, 0)) + +#define EDMA_TCD_CSR_START BIT(0) +#define EDMA_TCD_CSR_INT_MAJOR BIT(1) +#define EDMA_TCD_CSR_INT_HALF BIT(2) +#define EDMA_TCD_CSR_D_REQ BIT(3) +#define EDMA_TCD_CSR_E_SG BIT(4) +#define EDMA_TCD_CSR_E_LINK BIT(5) +#define EDMA_TCD_CSR_ACTIVE BIT(6) +#define EDMA_TCD_CSR_DONE BIT(7) + +struct fsl_edma_chan *to_fsl_edma_chan(struct dma_chan *chan) +{ + return container_of(chan, struct fsl_edma_chan, vchan.chan); +} +EXPORT_SYMBOL_GPL(to_fsl_edma_chan); + +struct fsl_edma_desc *to_fsl_edma_desc(struct virt_dma_desc *vd) +{ + return container_of(vd, struct fsl_edma_desc, vdesc); +} +EXPORT_SYMBOL_GPL(to_fsl_edma_desc); + +/* + * R/W functions for big- or little-endian registers: + * The eDMA controller's endian is independent of the CPU core's endian. + * For the big-endian IP module, the offset for 8-bit or 16-bit registers + * should also be swapped opposite to that in little-endian IP. + */ +u32 edma_readl(struct fsl_edma_engine *edma, + void __iomem *addr) +{ + if (edma->big_endian) + return ioread32be(addr); + else + return ioread32(addr); +} +EXPORT_SYMBOL_GPL(edma_readl); + +void edma_writeb(struct fsl_edma_engine *edma, u8 val, + void __iomem *addr) +{ + /* swap the reg offset for these in big-endian mode */ + if (edma->big_endian) + iowrite8(val, (void __iomem *)((unsigned long)addr ^ 0x3)); + else + iowrite8(val, addr); +} +EXPORT_SYMBOL_GPL(edma_writeb); + +void edma_writew(struct fsl_edma_engine *edma, u16 val, + void __iomem *addr) +{ + /* swap the reg offset for these in big-endian mode */ + if (edma->big_endian) + iowrite16be(val, (void __iomem *)((unsigned long)addr ^ 0x2)); + else + iowrite16(val, addr); +} +EXPORT_SYMBOL_GPL(edma_writew); + +void edma_writel(struct fsl_edma_engine *edma, u32 val, + void __iomem *addr) +{ + if (edma->big_endian) + iowrite32be(val, addr); + else + iowrite32(val, addr); +} +EXPORT_SYMBOL_GPL(edma_writel); + +static void fsl_edma_fill_tcd(struct fsl_edma_hw_tcd *tcd, + u32 src, u32 dst, u16 attr, u16 soff, + u32 nbytes, u32 slast, u16 citer, u16 biter, + u16 doff, u32 dlast_sga, bool major_int, + bool disable_req, bool enable_sg) +{ + u16 csr = 0; + + /* + * eDMA hardware SGs require the TCDs to be stored in little + * endian format irrespective of the register endian model. + * So we put the value in little endian in memory, waiting + * for fsl_edma_set_tcd_regs doing the swap. + */ + tcd->saddr = cpu_to_le32(src); + tcd->daddr = cpu_to_le32(dst); + tcd->attr = cpu_to_le16(attr); + tcd->soff = cpu_to_le16(soff); + tcd->nbytes = cpu_to_le32(nbytes); + tcd->slast = cpu_to_le32(slast); + tcd->citer = cpu_to_le16(EDMA_TCD_CITER_CITER(citer)); + tcd->doff = cpu_to_le16(doff); + tcd->dlast_sga = cpu_to_le32(dlast_sga); + tcd->biter = cpu_to_le16(EDMA_TCD_BITER_BITER(biter)); + + if (major_int) + csr |= EDMA_TCD_CSR_INT_MAJOR; + if (disable_req) + csr |= EDMA_TCD_CSR_D_REQ; + if (enable_sg) + csr |= EDMA_TCD_CSR_E_SG; + + tcd->csr = cpu_to_le16(csr); +} + +static void fsl_edma_enable_request(struct fsl_edma_chan *fsl_chan) +{ + struct edma_regs *regs = &fsl_chan->edma->regs; + u32 ch = fsl_chan->vchan.chan.chan_id; + + if (fsl_chan->edma->version == v1) { + edma_writeb(fsl_chan->edma, EDMA_SEEI_SEEI(ch), regs->seei); + edma_writeb(fsl_chan->edma, ch, regs->serq); + } else { + /* ColdFire is big endian, and accesses natively + * big endian I/O peripherals + */ + iowrite8(EDMA_SEEI_SEEI(ch), regs->seei); + iowrite8(ch, regs->serq); + } +} + +static unsigned int fsl_edma_get_tcd_attr(enum dma_slave_buswidth addr_width) +{ + switch (addr_width) { + case 1: + return EDMA_TCD_ATTR_SSIZE_8BIT | EDMA_TCD_ATTR_DSIZE_8BIT; + case 2: + return EDMA_TCD_ATTR_SSIZE_16BIT | EDMA_TCD_ATTR_DSIZE_16BIT; + case 4: + return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT; + case 8: + return EDMA_TCD_ATTR_SSIZE_64BIT | EDMA_TCD_ATTR_DSIZE_64BIT; + default: + return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT; + } +} + +static struct fsl_edma_desc *fsl_edma_alloc_desc(struct fsl_edma_chan *fsl_chan, + int sg_len) +{ + struct fsl_edma_desc *fsl_desc; + int i; + + fsl_desc = kzalloc(sizeof(*fsl_desc) + + sizeof(struct fsl_edma_sw_tcd) * sg_len, GFP_NOWAIT); + if (!fsl_desc) + return NULL; + + fsl_desc->echan = fsl_chan; + fsl_desc->n_tcds = sg_len; + for (i = 0; i < sg_len; i++) { + fsl_desc->tcd[i].vtcd = dma_pool_alloc(fsl_chan->tcd_pool, + GFP_NOWAIT, &fsl_desc->tcd[i].ptcd); + if (!fsl_desc->tcd[i].vtcd) + goto err; + } + return fsl_desc; + +err: + while (--i >= 0) + dma_pool_free(fsl_chan->tcd_pool, fsl_desc->tcd[i].vtcd, + fsl_desc->tcd[i].ptcd); + kfree(fsl_desc); + return NULL; +} + +static void fsl_edma_set_tcd_regs(struct fsl_edma_chan *fsl_chan, + struct fsl_edma_hw_tcd *tcd) +{ + struct fsl_edma_engine *edma = fsl_chan->edma; + struct fsl_edma_hw_tcd __iomem *mtcd = edma->regs.tcd + + fsl_chan->vchan.chan.chan_id; + + /* + * TCD parameters are stored in struct fsl_edma_hw_tcd in little + * endian format. However, we need to load the TCD registers in + * big- or little-endian obeying the eDMA engine model endian. + */ + edma_writel(edma, le32_to_cpu(tcd->saddr), &mtcd->saddr); + edma_writel(edma, le32_to_cpu(tcd->daddr), &mtcd->daddr); + edma_writew(edma, le16_to_cpu(tcd->attr), &mtcd->attr); + edma_writew(edma, le16_to_cpu(tcd->soff), &mtcd->soff); + edma_writel(edma, le32_to_cpu(tcd->nbytes), &mtcd->nbytes); + edma_writel(edma, le32_to_cpu(tcd->slast), &mtcd->slast); + edma_writew(edma, le16_to_cpu(tcd->citer), &mtcd->citer); + edma_writew(edma, le16_to_cpu(tcd->biter), &mtcd->biter); + edma_writew(edma, le16_to_cpu(tcd->doff), &mtcd->doff); + edma_writel(edma, le32_to_cpu(tcd->dlast_sga), &mtcd->dlast_sga); + edma_writew(edma, le16_to_cpu(tcd->csr), &mtcd->csr); +} + +static size_t fsl_edma_desc_residue(struct fsl_edma_chan *fsl_chan, + struct virt_dma_desc *vdesc, bool in_progress) +{ + struct fsl_edma_desc *edesc = fsl_chan->edesc; + struct edma_regs *regs = &fsl_chan->edma->regs; + u32 ch = fsl_chan->vchan.chan.chan_id; + enum dma_transfer_direction dir = fsl_chan->dir; + dma_addr_t cur_addr, dma_addr; + size_t len, size; + int i; + + /* calculate the total size in this desc */ + for (len = i = 0; i < fsl_chan->edesc->n_tcds; i++) + len += le32_to_cpu(edesc->tcd[i].vtcd->nbytes) + * le16_to_cpu(edesc->tcd[i].vtcd->biter); + + if (!in_progress) + return len; + + if (dir == DMA_MEM_TO_DEV) + cur_addr = edma_readl(fsl_chan->edma, ®s->tcd[ch].saddr); + else + cur_addr = edma_readl(fsl_chan->edma, ®s->tcd[ch].daddr); + + /* figure out the finished and calculate the residue */ + for (i = 0; i < fsl_chan->edesc->n_tcds; i++) { + size = le32_to_cpu(edesc->tcd[i].vtcd->nbytes) + * le16_to_cpu(edesc->tcd[i].vtcd->biter); + if (dir == DMA_MEM_TO_DEV) + dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->saddr); + else + dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->daddr); + + len -= size; + if (cur_addr >= dma_addr && cur_addr < dma_addr + size) { + len += dma_addr + size - cur_addr; + break; + } + } + + return len; +} + +void fsl_edma_disable_request(struct fsl_edma_chan *fsl_chan) +{ + struct edma_regs *regs = &fsl_chan->edma->regs; + u32 ch = fsl_chan->vchan.chan.chan_id; + + if (fsl_chan->edma->version == v1) { + edma_writeb(fsl_chan->edma, ch, regs->cerq); + edma_writeb(fsl_chan->edma, EDMA_CEEI_CEEI(ch), regs->ceei); + } else { + /* ColdFire is big endian, and accesses natively + * big endian I/O peripherals + */ + iowrite8(ch, regs->cerq); + iowrite8(EDMA_CEEI_CEEI(ch), regs->ceei); + } +} +EXPORT_SYMBOL_GPL(fsl_edma_disable_request); + +void fsl_edma_cleanup_vchan(struct dma_device *dmadev) +{ + struct fsl_edma_chan *chan, *_chan; + + list_for_each_entry_safe(chan, _chan, + &dmadev->channels, vchan.chan.device_node) { + list_del(&chan->vchan.chan.device_node); + tasklet_kill(&chan->vchan.task); + } +} +EXPORT_SYMBOL_GPL(fsl_edma_cleanup_vchan); + +void fsl_edma_free_desc(struct virt_dma_desc *vdesc) +{ + struct fsl_edma_desc *mcf_desc; + int i; + + mcf_desc = to_fsl_edma_desc(vdesc); + + for (i = 0; i < mcf_desc->n_tcds; i++) + dma_pool_free(mcf_desc->echan->tcd_pool, mcf_desc->tcd[i].vtcd, + mcf_desc->tcd[i].ptcd); + kfree(mcf_desc); +} +EXPORT_SYMBOL_GPL(fsl_edma_free_desc); + +int fsl_edma_alloc_chan_resources(struct dma_chan *chan) +{ + struct fsl_edma_chan *mcf_chan = to_fsl_edma_chan(chan); + + mcf_chan->tcd_pool = dma_pool_create("tcd_pool", chan->device->dev, + sizeof(struct fsl_edma_hw_tcd), EDMA_TCD_MEM_ALIGN, 0); + + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_alloc_chan_resources); + +void fsl_edma_free_chan_resources(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + unsigned long flags; + LIST_HEAD(head); + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + fsl_edma_disable_request(fsl_chan); + fsl_chan->edesc = NULL; + vchan_get_all_descriptors(&fsl_chan->vchan, &head); + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + + vchan_dma_desc_free_list(&fsl_chan->vchan, &head); + dma_pool_destroy(fsl_chan->tcd_pool); + fsl_chan->tcd_pool = NULL; +} +EXPORT_SYMBOL_GPL(fsl_edma_free_chan_resources); + +int fsl_edma_slave_config(struct dma_chan *chan, + struct dma_slave_config *config) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + + memcpy(&fsl_chan->cfg, config, sizeof(*config)); + + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_slave_config); + +struct dma_async_tx_descriptor *fsl_edma_prep_dma_cyclic( + struct dma_chan *chan, dma_addr_t dma_addr, size_t buf_len, + size_t period_len, enum dma_transfer_direction direction, + unsigned long flags) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + struct fsl_edma_desc *fsl_desc; + dma_addr_t dma_buf_next; + int sg_len, i; + u32 src_addr, dst_addr, last_sg, nbytes, attr; + u16 soff, doff, iter; + + if (!is_slave_direction(direction)) + return NULL; + + sg_len = buf_len / period_len; + fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len); + if (!fsl_desc) + return NULL; + + fsl_desc->iscyclic = true; + fsl_chan->dir = direction; + + dma_buf_next = dma_addr; + + for (i = 0; i < sg_len; i++) { + if (dma_buf_next >= dma_addr + buf_len) + dma_buf_next = dma_addr; + + /* get next sg's physical address */ + last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd; + + if (direction == DMA_MEM_TO_DEV) { + nbytes = fsl_chan->cfg.dst_addr_width * + fsl_chan->cfg.dst_maxburst; + src_addr = dma_buf_next; + dst_addr = fsl_chan->cfg.dst_addr; + soff = fsl_chan->cfg.dst_addr_width; + doff = 0; + attr = fsl_edma_get_tcd_attr( + fsl_chan->cfg.dst_addr_width); + } else { + nbytes = fsl_chan->cfg.src_addr_width * + fsl_chan->cfg.src_maxburst; + src_addr = fsl_chan->cfg.src_addr; + dst_addr = dma_buf_next; + soff = 0; + doff = fsl_chan->cfg.src_addr_width; + attr = fsl_edma_get_tcd_attr( + fsl_chan->cfg.src_addr_width); + } + + iter = period_len / nbytes; + + fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, + src_addr, dst_addr, + attr, soff, nbytes, 0, iter, + iter, doff, last_sg, true, false, true); + dma_buf_next += period_len; + } + + return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags); +} +EXPORT_SYMBOL_GPL(fsl_edma_prep_dma_cyclic); + +struct dma_async_tx_descriptor *fsl_edma_prep_slave_sg( + struct dma_chan *chan, struct scatterlist *sgl, + unsigned int sg_len, enum dma_transfer_direction direction, + unsigned long flags, void *context) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + struct fsl_edma_desc *fsl_desc; + struct scatterlist *sg; + u32 src_addr, dst_addr, last_sg, nbytes, attr; + u16 soff, doff, iter; + int i; + + if (!is_slave_direction(direction)) + return NULL; + + fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len); + if (!fsl_desc) + return NULL; + + fsl_desc->iscyclic = false; + fsl_chan->dir = direction; + + for_each_sg(sgl, sg, sg_len, i) { + /* get next sg's physical address */ + last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd; + + if (direction == DMA_MEM_TO_DEV) { + nbytes = fsl_chan->cfg.dst_addr_width * + fsl_chan->cfg.dst_maxburst; + src_addr = sg_dma_address(sg); + dst_addr = fsl_chan->cfg.dst_addr; + soff = fsl_chan->cfg.dst_addr_width; + doff = 0; + attr = fsl_edma_get_tcd_attr( + fsl_chan->cfg.dst_addr_width); + + } else { + nbytes = fsl_chan->cfg.src_addr_width * + fsl_chan->cfg.src_maxburst; + src_addr = fsl_chan->cfg.src_addr; + dst_addr = sg_dma_address(sg); + soff = 0; + doff = fsl_chan->cfg.src_addr_width; + attr = fsl_edma_get_tcd_attr( + fsl_chan->cfg.src_addr_width); + } + + iter = sg_dma_len(sg) / nbytes; + + if (i < sg_len - 1) { + last_sg = fsl_desc->tcd[(i + 1)].ptcd; + fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, + src_addr, + dst_addr, attr, soff, + nbytes, 0, iter, iter, doff, last_sg, + false, false, true); + } else { + last_sg = 0; + fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, + src_addr, + dst_addr, attr, soff, + nbytes, 0, iter, iter, doff, last_sg, + true, true, false); + } + } + + return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags); +} +EXPORT_SYMBOL_GPL(fsl_edma_prep_slave_sg); + +enum dma_status fsl_edma_tx_status(struct dma_chan *chan, + dma_cookie_t cookie, struct dma_tx_state *txstate) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + struct virt_dma_desc *vdesc; + enum dma_status status; + unsigned long flags; + + status = dma_cookie_status(chan, cookie, txstate); + if (status == DMA_COMPLETE) + return status; + + if (!txstate) + return fsl_chan->status; + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + vdesc = vchan_find_desc(&fsl_chan->vchan, cookie); + if (fsl_chan->edesc && cookie == fsl_chan->edesc->vdesc.tx.cookie) + txstate->residue = + fsl_edma_desc_residue(fsl_chan, vdesc, true); + else if (vdesc) + txstate->residue = + fsl_edma_desc_residue(fsl_chan, vdesc, false); + else + txstate->residue = 0; + + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + + return fsl_chan->status; +} +EXPORT_SYMBOL_GPL(fsl_edma_tx_status); + +int fsl_edma_pause(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + unsigned long flags; + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + if (fsl_chan->edesc) { + fsl_edma_disable_request(fsl_chan); + fsl_chan->status = DMA_PAUSED; + fsl_chan->idle = true; + } + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_pause); + +int fsl_edma_resume(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + unsigned long flags; + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + if (fsl_chan->edesc) { + fsl_edma_enable_request(fsl_chan); + fsl_chan->status = DMA_IN_PROGRESS; + fsl_chan->idle = false; + } + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_resume); + +int fsl_edma_terminate_all(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + unsigned long flags; + LIST_HEAD(head); + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + fsl_edma_disable_request(fsl_chan); + fsl_chan->edesc = NULL; + fsl_chan->idle = true; + vchan_get_all_descriptors(&fsl_chan->vchan, &head); + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + vchan_dma_desc_free_list(&fsl_chan->vchan, &head); + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_terminate_all); + +void fsl_edma_issue_pending(struct dma_chan *chan) +{ + struct fsl_edma_chan *mcf_chan = to_fsl_edma_chan(chan); + unsigned long flags; + + spin_lock_irqsave(&mcf_chan->vchan.lock, flags); + + if (vchan_issue_pending(&mcf_chan->vchan) && !mcf_chan->edesc) + fsl_edma_xfer_desc(mcf_chan); + + spin_unlock_irqrestore(&mcf_chan->vchan.lock, flags); +} +EXPORT_SYMBOL_GPL(fsl_edma_issue_pending); + +void fsl_edma_xfer_desc(struct fsl_edma_chan *fsl_chan) +{ + struct virt_dma_desc *vdesc; + + vdesc = vchan_next_desc(&fsl_chan->vchan); + if (!vdesc) + return; + fsl_chan->edesc = to_fsl_edma_desc(vdesc); + fsl_edma_set_tcd_regs(fsl_chan, fsl_chan->edesc->tcd[0].vtcd); + fsl_edma_enable_request(fsl_chan); + fsl_chan->status = DMA_IN_PROGRESS; + fsl_chan->idle = false; +} +EXPORT_SYMBOL_GPL(fsl_edma_xfer_desc); + +/* + * On the 32 channels Vybrid/mpc577x edma version (here called "v1"), + * register offsets are different compared to ColdFire mcf5441x 64 channels + * edma (here called "v2"). + * + * This function sets up register offsets as per proper declared version + * so must be called in xxx_edma_probe() just after setting the + * edma "version" and "membase" appropriately. + */ +void fsl_edma_setup_regs(struct fsl_edma_engine *edma) +{ + edma->regs.cr = edma->membase + EDMA_CR; + edma->regs.es = edma->membase + EDMA_ES; + edma->regs.erql = edma->membase + EDMA_ERQ; + edma->regs.eeil = edma->membase + EDMA_EEI; + + edma->regs.serq = edma->membase + ((edma->version == v1) ? + EDMA_SERQ : EDMA64_SERQ); + edma->regs.cerq = edma->membase + ((edma->version == v1) ? + EDMA_CERQ : EDMA64_CERQ); + edma->regs.seei = edma->membase + ((edma->version == v1) ? + EDMA_SEEI : EDMA64_SEEI); + edma->regs.ceei = edma->membase + ((edma->version == v1) ? + EDMA_CEEI : EDMA64_CEEI); + edma->regs.cint = edma->membase + ((edma->version == v1) ? + EDMA_CINT : EDMA64_CINT); + edma->regs.cerr = edma->membase + ((edma->version == v1) ? + EDMA_CERR : EDMA64_CERR); + edma->regs.ssrt = edma->membase + ((edma->version == v1) ? + EDMA_SSRT : EDMA64_SSRT); + edma->regs.cdne = edma->membase + ((edma->version == v1) ? + EDMA_CDNE : EDMA64_CDNE); + edma->regs.intl = edma->membase + ((edma->version == v1) ? + EDMA_INTR : EDMA64_INTL); + edma->regs.errl = edma->membase + ((edma->version == v1) ? + EDMA_ERR : EDMA64_ERRL); + + if (edma->version == v2) { + edma->regs.erqh = edma->membase + EDMA64_ERQH; + edma->regs.eeih = edma->membase + EDMA64_EEIH; + edma->regs.errh = edma->membase + EDMA64_ERRH; + edma->regs.inth = edma->membase + EDMA64_INTH; + } + + edma->regs.tcd = edma->membase + EDMA_TCD; +} +EXPORT_SYMBOL_GPL(fsl_edma_setup_regs); diff --git a/drivers/dma/fsl-edma-common.h b/drivers/dma/fsl-edma-common.h new file mode 100644 index 000000000000..8e3c609c62fa --- /dev/null +++ b/drivers/dma/fsl-edma-common.h @@ -0,0 +1,161 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright 2013-2014 Freescale Semiconductor, Inc. + * Copyright 2018 Angelo Dureghello + */ +#ifndef _FSL_EDMA_COMMON_H_ +#define _FSL_EDMA_COMMON_H_ + +#include + +#include "virt-dma.h" + +#define DMAMUX_NR 2 + +#define EDMA_CR_EDBG BIT(1) +#define EDMA_CR_ERCA BIT(2) +#define EDMA_CR_ERGA BIT(3) +#define EDMA_CR_HOE BIT(4) +#define EDMA_CR_HALT BIT(5) +#define EDMA_CR_CLM BIT(6) +#define EDMA_CR_EMLM BIT(7) +#define EDMA_CR_ECX BIT(16) +#define EDMA_CR_CX BIT(17) + +#define EDMA_SEEI_SEEI(x) ((x) & GENMASK(6, 0)) +#define EDMA_CEEI_CEEI(x) ((x) & GENMASK(6, 0)) +#define EDMA_CINT_CINT(x) ((x) & GENMASK(6, 0)) +#define EDMA_CERR_CERR(x) ((x) & GENMASK(6, 0)) + +#define FSL_EDMA_BUSWIDTHS (BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) | \ + BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) | \ + BIT(DMA_SLAVE_BUSWIDTH_4_BYTES) | \ + BIT(DMA_SLAVE_BUSWIDTH_8_BYTES)) + +enum fsl_edma_pm_state { + RUNNING = 0, + SUSPENDED, +}; + +/* + * This are tcd regs, equal for both v32 and v64 + */ +struct fsl_edma_hw_tcd { + __le32 saddr; + __le16 soff; + __le16 attr; + __le32 nbytes; + __le32 slast; + __le32 daddr; + __le16 doff; + __le16 citer; + __le32 dlast_sga; + __le16 csr; + __le16 biter; +}; + +/* + * This are iomem pointers, for both v32 and v64. + */ +struct edma_regs { + void __iomem *cr; + void __iomem *es; + void __iomem *erq; + void __iomem *erqh; + void __iomem *erql; /* aka erq on v32 */ + void __iomem *eeih; + void __iomem *eeil; /* aka eei on v32 */ + void __iomem *seei; + void __iomem *ceei; + void __iomem *serq; + void __iomem *cerq; + void __iomem *cint; + void __iomem *cerr; + void __iomem *ssrt; + void __iomem *cdne; + void __iomem *inth; + void __iomem *intl; + void __iomem *errh; + void __iomem *errl; + struct fsl_edma_hw_tcd __iomem *tcd; +}; + +struct fsl_edma_sw_tcd { + dma_addr_t ptcd; + struct fsl_edma_hw_tcd *vtcd; +}; + +struct fsl_edma_chan { + struct virt_dma_chan vchan; + enum dma_status status; + enum fsl_edma_pm_state pm_state; + bool idle; + u32 slave_id; + struct fsl_edma_engine *edma; + struct fsl_edma_desc *edesc; + struct dma_pool *tcd_pool; + struct dma_slave_config cfg; + enum dma_transfer_direction dir; +}; + +struct fsl_edma_desc { + struct virt_dma_desc vdesc; + struct fsl_edma_chan *echan; + bool iscyclic; + unsigned int n_tcds; + struct fsl_edma_sw_tcd tcd[]; +}; + +enum edma_version { + v1, /* 32ch, Vybdir, mpc57x, etc */ + v2, /* 64ch Coldfire */ +}; + +struct fsl_edma_engine { + struct dma_device dma_dev; + struct edma_regs regs; + void __iomem *membase; + void __iomem *muxbase[DMAMUX_NR]; + struct clk *muxclk[DMAMUX_NR]; + struct mutex fsl_edma_mutex; + u32 n_chans; + int txirq; + int errirq; + bool big_endian; + enum edma_version version; + struct fsl_edma_chan chans[]; +}; + +u32 edma_readl(struct fsl_edma_engine *edma, void __iomem *addr); +void edma_writeb(struct fsl_edma_engine *edma, u8 val, void __iomem *addr); +void edma_writew(struct fsl_edma_engine *edma, u16 val, void __iomem *addr); +void edma_writel(struct fsl_edma_engine *edma, u32 val, void __iomem *addr); +struct fsl_edma_chan *to_fsl_edma_chan(struct dma_chan *chan); +struct fsl_edma_desc *to_fsl_edma_desc(struct virt_dma_desc *vd); +void fsl_edma_setup_regs(struct fsl_edma_engine *edma); +void fsl_edma_disable_request(struct fsl_edma_chan *fsl_chan); +void fsl_edma_free_desc(struct virt_dma_desc *vdesc); +void fsl_edma_xfer_desc(struct fsl_edma_chan *fsl_chan); +void fsl_edma_cleanup_vchan(struct dma_device *dmadev); + +/* Operations */ +int fsl_edma_alloc_chan_resources(struct dma_chan *chan); +void fsl_edma_free_chan_resources(struct dma_chan *chan); +int fsl_edma_slave_config(struct dma_chan *chan, + struct dma_slave_config *config); +struct dma_async_tx_descriptor *fsl_edma_prep_dma_cyclic( + struct dma_chan *chan, dma_addr_t dma_addr, size_t buf_len, + size_t period_len, enum dma_transfer_direction direction, + unsigned long flags); +struct dma_async_tx_descriptor *fsl_edma_prep_slave_sg( + struct dma_chan *chan, struct scatterlist *sgl, + unsigned int sg_len, enum dma_transfer_direction direction, + unsigned long flags, void *context); +enum dma_status fsl_edma_tx_status(struct dma_chan *chan, + dma_cookie_t cookie, struct dma_tx_state *txstate); +int fsl_edma_pause(struct dma_chan *chan); +int fsl_edma_resume(struct dma_chan *chan); +int fsl_edma_terminate_all(struct dma_chan *chan); +void fsl_edma_issue_pending(struct dma_chan *chan); + +#endif /* _FSL_EDMA_COMMON_H_ */ diff --git a/drivers/dma/fsl-edma.c b/drivers/dma/fsl-edma.c index c7568869284e..5a4166e485f9 100644 --- a/drivers/dma/fsl-edma.c +++ b/drivers/dma/fsl-edma.c @@ -1,254 +1,21 @@ -/* - * drivers/dma/fsl-edma.c - * - * Copyright 2013-2014 Freescale Semiconductor, Inc. - * - * Driver for the Freescale eDMA engine with flexible channel multiplexing - * capability for DMA request sources. The eDMA block can be found on some - * Vybrid and Layerscape SoCs. - * - * This program is free software; you can redistribute it and/or modify it - * under the terms of the GNU General Public License as published by the - * Free Software Foundation; either version 2 of the License, or (at your - * option) any later version. - */ +// SPDX-License-Identifier: GPL-2.0 +// +// Copyright 2013-2014 Freescale Semiconductor, Inc. -#include #include #include #include -#include -#include -#include -#include #include #include #include #include #include -#include "virt-dma.h" - -#define EDMA_CR 0x00 -#define EDMA_ES 0x04 -#define EDMA_ERQ 0x0C -#define EDMA_EEI 0x14 -#define EDMA_SERQ 0x1B -#define EDMA_CERQ 0x1A -#define EDMA_SEEI 0x19 -#define EDMA_CEEI 0x18 -#define EDMA_CINT 0x1F -#define EDMA_CERR 0x1E -#define EDMA_SSRT 0x1D -#define EDMA_CDNE 0x1C -#define EDMA_INTR 0x24 -#define EDMA_ERR 0x2C - -#define EDMA_TCD_SADDR(x) (0x1000 + 32 * (x)) -#define EDMA_TCD_SOFF(x) (0x1004 + 32 * (x)) -#define EDMA_TCD_ATTR(x) (0x1006 + 32 * (x)) -#define EDMA_TCD_NBYTES(x) (0x1008 + 32 * (x)) -#define EDMA_TCD_SLAST(x) (0x100C + 32 * (x)) -#define EDMA_TCD_DADDR(x) (0x1010 + 32 * (x)) -#define EDMA_TCD_DOFF(x) (0x1014 + 32 * (x)) -#define EDMA_TCD_CITER_ELINK(x) (0x1016 + 32 * (x)) -#define EDMA_TCD_CITER(x) (0x1016 + 32 * (x)) -#define EDMA_TCD_DLAST_SGA(x) (0x1018 + 32 * (x)) -#define EDMA_TCD_CSR(x) (0x101C + 32 * (x)) -#define EDMA_TCD_BITER_ELINK(x) (0x101E + 32 * (x)) -#define EDMA_TCD_BITER(x) (0x101E + 32 * (x)) - -#define EDMA_CR_EDBG BIT(1) -#define EDMA_CR_ERCA BIT(2) -#define EDMA_CR_ERGA BIT(3) -#define EDMA_CR_HOE BIT(4) -#define EDMA_CR_HALT BIT(5) -#define EDMA_CR_CLM BIT(6) -#define EDMA_CR_EMLM BIT(7) -#define EDMA_CR_ECX BIT(16) -#define EDMA_CR_CX BIT(17) - -#define EDMA_SEEI_SEEI(x) ((x) & 0x1F) -#define EDMA_CEEI_CEEI(x) ((x) & 0x1F) -#define EDMA_CINT_CINT(x) ((x) & 0x1F) -#define EDMA_CERR_CERR(x) ((x) & 0x1F) - -#define EDMA_TCD_ATTR_DSIZE(x) (((x) & 0x0007)) -#define EDMA_TCD_ATTR_DMOD(x) (((x) & 0x001F) << 3) -#define EDMA_TCD_ATTR_SSIZE(x) (((x) & 0x0007) << 8) -#define EDMA_TCD_ATTR_SMOD(x) (((x) & 0x001F) << 11) -#define EDMA_TCD_ATTR_SSIZE_8BIT (0x0000) -#define EDMA_TCD_ATTR_SSIZE_16BIT (0x0100) -#define EDMA_TCD_ATTR_SSIZE_32BIT (0x0200) -#define EDMA_TCD_ATTR_SSIZE_64BIT (0x0300) -#define EDMA_TCD_ATTR_SSIZE_32BYTE (0x0500) -#define EDMA_TCD_ATTR_DSIZE_8BIT (0x0000) -#define EDMA_TCD_ATTR_DSIZE_16BIT (0x0001) -#define EDMA_TCD_ATTR_DSIZE_32BIT (0x0002) -#define EDMA_TCD_ATTR_DSIZE_64BIT (0x0003) -#define EDMA_TCD_ATTR_DSIZE_32BYTE (0x0005) - -#define EDMA_TCD_SOFF_SOFF(x) (x) -#define EDMA_TCD_NBYTES_NBYTES(x) (x) -#define EDMA_TCD_SLAST_SLAST(x) (x) -#define EDMA_TCD_DADDR_DADDR(x) (x) -#define EDMA_TCD_CITER_CITER(x) ((x) & 0x7FFF) -#define EDMA_TCD_DOFF_DOFF(x) (x) -#define EDMA_TCD_DLAST_SGA_DLAST_SGA(x) (x) -#define EDMA_TCD_BITER_BITER(x) ((x) & 0x7FFF) - -#define EDMA_TCD_CSR_START BIT(0) -#define EDMA_TCD_CSR_INT_MAJOR BIT(1) -#define EDMA_TCD_CSR_INT_HALF BIT(2) -#define EDMA_TCD_CSR_D_REQ BIT(3) -#define EDMA_TCD_CSR_E_SG BIT(4) -#define EDMA_TCD_CSR_E_LINK BIT(5) -#define EDMA_TCD_CSR_ACTIVE BIT(6) -#define EDMA_TCD_CSR_DONE BIT(7) - -#define EDMAMUX_CHCFG_DIS 0x0 -#define EDMAMUX_CHCFG_ENBL 0x80 -#define EDMAMUX_CHCFG_SOURCE(n) ((n) & 0x3F) - -#define DMAMUX_NR 2 - -#define FSL_EDMA_BUSWIDTHS BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) | \ - BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) | \ - BIT(DMA_SLAVE_BUSWIDTH_4_BYTES) | \ - BIT(DMA_SLAVE_BUSWIDTH_8_BYTES) -enum fsl_edma_pm_state { - RUNNING = 0, - SUSPENDED, -}; - -struct fsl_edma_hw_tcd { - __le32 saddr; - __le16 soff; - __le16 attr; - __le32 nbytes; - __le32 slast; - __le32 daddr; - __le16 doff; - __le16 citer; - __le32 dlast_sga; - __le16 csr; - __le16 biter; -}; - -struct fsl_edma_sw_tcd { - dma_addr_t ptcd; - struct fsl_edma_hw_tcd *vtcd; -}; - -struct fsl_edma_slave_config { - enum dma_transfer_direction dir; - enum dma_slave_buswidth addr_width; - u32 dev_addr; - u32 burst; - u32 attr; -}; - -struct fsl_edma_chan { - struct virt_dma_chan vchan; - enum dma_status status; - enum fsl_edma_pm_state pm_state; - bool idle; - u32 slave_id; - struct fsl_edma_engine *edma; - struct fsl_edma_desc *edesc; - struct fsl_edma_slave_config fsc; - struct dma_pool *tcd_pool; -}; - -struct fsl_edma_desc { - struct virt_dma_desc vdesc; - struct fsl_edma_chan *echan; - bool iscyclic; - unsigned int n_tcds; - struct fsl_edma_sw_tcd tcd[]; -}; - -struct fsl_edma_engine { - struct dma_device dma_dev; - void __iomem *membase; - void __iomem *muxbase[DMAMUX_NR]; - struct clk *muxclk[DMAMUX_NR]; - struct mutex fsl_edma_mutex; - u32 n_chans; - int txirq; - int errirq; - bool big_endian; - struct fsl_edma_chan chans[]; -}; - -/* - * R/W functions for big- or little-endian registers: - * The eDMA controller's endian is independent of the CPU core's endian. - * For the big-endian IP module, the offset for 8-bit or 16-bit registers - * should also be swapped opposite to that in little-endian IP. - */ - -static u32 edma_readl(struct fsl_edma_engine *edma, void __iomem *addr) -{ - if (edma->big_endian) - return ioread32be(addr); - else - return ioread32(addr); -} - -static void edma_writeb(struct fsl_edma_engine *edma, u8 val, void __iomem *addr) -{ - /* swap the reg offset for these in big-endian mode */ - if (edma->big_endian) - iowrite8(val, (void __iomem *)((unsigned long)addr ^ 0x3)); - else - iowrite8(val, addr); -} - -static void edma_writew(struct fsl_edma_engine *edma, u16 val, void __iomem *addr) -{ - /* swap the reg offset for these in big-endian mode */ - if (edma->big_endian) - iowrite16be(val, (void __iomem *)((unsigned long)addr ^ 0x2)); - else - iowrite16(val, addr); -} +#include "fsl-edma-common.h" -static void edma_writel(struct fsl_edma_engine *edma, u32 val, void __iomem *addr) -{ - if (edma->big_endian) - iowrite32be(val, addr); - else - iowrite32(val, addr); -} - -static struct fsl_edma_chan *to_fsl_edma_chan(struct dma_chan *chan) -{ - return container_of(chan, struct fsl_edma_chan, vchan.chan); -} - -static struct fsl_edma_desc *to_fsl_edma_desc(struct virt_dma_desc *vd) -{ - return container_of(vd, struct fsl_edma_desc, vdesc); -} - -static void fsl_edma_enable_request(struct fsl_edma_chan *fsl_chan) -{ - void __iomem *addr = fsl_chan->edma->membase; - u32 ch = fsl_chan->vchan.chan.chan_id; - - edma_writeb(fsl_chan->edma, EDMA_SEEI_SEEI(ch), addr + EDMA_SEEI); - edma_writeb(fsl_chan->edma, ch, addr + EDMA_SERQ); -} - -static void fsl_edma_disable_request(struct fsl_edma_chan *fsl_chan) -{ - void __iomem *addr = fsl_chan->edma->membase; - u32 ch = fsl_chan->vchan.chan.chan_id; - - edma_writeb(fsl_chan->edma, ch, addr + EDMA_CERQ); - edma_writeb(fsl_chan->edma, EDMA_CEEI_CEEI(ch), addr + EDMA_CEEI); -} +#define EDMAMUX_CHCFG_DIS 0 +#define EDMAMUX_CHCFG_ENBL BIT(7) +#define EDMAMUX_CHCFG_SOURCE(n) ((n) & GENMASK(6, 0)) static void fsl_edma_chan_mux(struct fsl_edma_chan *fsl_chan, unsigned int slot, bool enable) @@ -268,416 +35,20 @@ static void fsl_edma_chan_mux(struct fsl_edma_chan *fsl_chan, iowrite8(EDMAMUX_CHCFG_DIS, muxaddr + ch_off); } -static unsigned int fsl_edma_get_tcd_attr(enum dma_slave_buswidth addr_width) -{ - switch (addr_width) { - case 1: - return EDMA_TCD_ATTR_SSIZE_8BIT | EDMA_TCD_ATTR_DSIZE_8BIT; - case 2: - return EDMA_TCD_ATTR_SSIZE_16BIT | EDMA_TCD_ATTR_DSIZE_16BIT; - case 4: - return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT; - case 8: - return EDMA_TCD_ATTR_SSIZE_64BIT | EDMA_TCD_ATTR_DSIZE_64BIT; - default: - return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT; - } -} - -static void fsl_edma_free_desc(struct virt_dma_desc *vdesc) -{ - struct fsl_edma_desc *fsl_desc; - int i; - - fsl_desc = to_fsl_edma_desc(vdesc); - for (i = 0; i < fsl_desc->n_tcds; i++) - dma_pool_free(fsl_desc->echan->tcd_pool, fsl_desc->tcd[i].vtcd, - fsl_desc->tcd[i].ptcd); - kfree(fsl_desc); -} - -static int fsl_edma_terminate_all(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - LIST_HEAD(head); - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - fsl_edma_disable_request(fsl_chan); - fsl_chan->edesc = NULL; - fsl_chan->idle = true; - vchan_get_all_descriptors(&fsl_chan->vchan, &head); - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - vchan_dma_desc_free_list(&fsl_chan->vchan, &head); - return 0; -} - -static int fsl_edma_pause(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - if (fsl_chan->edesc) { - fsl_edma_disable_request(fsl_chan); - fsl_chan->status = DMA_PAUSED; - fsl_chan->idle = true; - } - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - return 0; -} - -static int fsl_edma_resume(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - if (fsl_chan->edesc) { - fsl_edma_enable_request(fsl_chan); - fsl_chan->status = DMA_IN_PROGRESS; - fsl_chan->idle = false; - } - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - return 0; -} - -static int fsl_edma_slave_config(struct dma_chan *chan, - struct dma_slave_config *cfg) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - - fsl_chan->fsc.dir = cfg->direction; - if (cfg->direction == DMA_DEV_TO_MEM) { - fsl_chan->fsc.dev_addr = cfg->src_addr; - fsl_chan->fsc.addr_width = cfg->src_addr_width; - fsl_chan->fsc.burst = cfg->src_maxburst; - fsl_chan->fsc.attr = fsl_edma_get_tcd_attr(cfg->src_addr_width); - } else if (cfg->direction == DMA_MEM_TO_DEV) { - fsl_chan->fsc.dev_addr = cfg->dst_addr; - fsl_chan->fsc.addr_width = cfg->dst_addr_width; - fsl_chan->fsc.burst = cfg->dst_maxburst; - fsl_chan->fsc.attr = fsl_edma_get_tcd_attr(cfg->dst_addr_width); - } else { - return -EINVAL; - } - return 0; -} - -static size_t fsl_edma_desc_residue(struct fsl_edma_chan *fsl_chan, - struct virt_dma_desc *vdesc, bool in_progress) -{ - struct fsl_edma_desc *edesc = fsl_chan->edesc; - void __iomem *addr = fsl_chan->edma->membase; - u32 ch = fsl_chan->vchan.chan.chan_id; - enum dma_transfer_direction dir = fsl_chan->fsc.dir; - dma_addr_t cur_addr, dma_addr; - size_t len, size; - int i; - - /* calculate the total size in this desc */ - for (len = i = 0; i < fsl_chan->edesc->n_tcds; i++) - len += le32_to_cpu(edesc->tcd[i].vtcd->nbytes) - * le16_to_cpu(edesc->tcd[i].vtcd->biter); - - if (!in_progress) - return len; - - if (dir == DMA_MEM_TO_DEV) - cur_addr = edma_readl(fsl_chan->edma, addr + EDMA_TCD_SADDR(ch)); - else - cur_addr = edma_readl(fsl_chan->edma, addr + EDMA_TCD_DADDR(ch)); - - /* figure out the finished and calculate the residue */ - for (i = 0; i < fsl_chan->edesc->n_tcds; i++) { - size = le32_to_cpu(edesc->tcd[i].vtcd->nbytes) - * le16_to_cpu(edesc->tcd[i].vtcd->biter); - if (dir == DMA_MEM_TO_DEV) - dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->saddr); - else - dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->daddr); - - len -= size; - if (cur_addr >= dma_addr && cur_addr < dma_addr + size) { - len += dma_addr + size - cur_addr; - break; - } - } - - return len; -} - -static enum dma_status fsl_edma_tx_status(struct dma_chan *chan, - dma_cookie_t cookie, struct dma_tx_state *txstate) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - struct virt_dma_desc *vdesc; - enum dma_status status; - unsigned long flags; - - status = dma_cookie_status(chan, cookie, txstate); - if (status == DMA_COMPLETE) - return status; - - if (!txstate) - return fsl_chan->status; - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - vdesc = vchan_find_desc(&fsl_chan->vchan, cookie); - if (fsl_chan->edesc && cookie == fsl_chan->edesc->vdesc.tx.cookie) - txstate->residue = fsl_edma_desc_residue(fsl_chan, vdesc, true); - else if (vdesc) - txstate->residue = fsl_edma_desc_residue(fsl_chan, vdesc, false); - else - txstate->residue = 0; - - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - - return fsl_chan->status; -} - -static void fsl_edma_set_tcd_regs(struct fsl_edma_chan *fsl_chan, - struct fsl_edma_hw_tcd *tcd) -{ - struct fsl_edma_engine *edma = fsl_chan->edma; - void __iomem *addr = fsl_chan->edma->membase; - u32 ch = fsl_chan->vchan.chan.chan_id; - - /* - * TCD parameters are stored in struct fsl_edma_hw_tcd in little - * endian format. However, we need to load the TCD registers in - * big- or little-endian obeying the eDMA engine model endian. - */ - edma_writew(edma, 0, addr + EDMA_TCD_CSR(ch)); - edma_writel(edma, le32_to_cpu(tcd->saddr), addr + EDMA_TCD_SADDR(ch)); - edma_writel(edma, le32_to_cpu(tcd->daddr), addr + EDMA_TCD_DADDR(ch)); - - edma_writew(edma, le16_to_cpu(tcd->attr), addr + EDMA_TCD_ATTR(ch)); - edma_writew(edma, le16_to_cpu(tcd->soff), addr + EDMA_TCD_SOFF(ch)); - - edma_writel(edma, le32_to_cpu(tcd->nbytes), addr + EDMA_TCD_NBYTES(ch)); - edma_writel(edma, le32_to_cpu(tcd->slast), addr + EDMA_TCD_SLAST(ch)); - - edma_writew(edma, le16_to_cpu(tcd->citer), addr + EDMA_TCD_CITER(ch)); - edma_writew(edma, le16_to_cpu(tcd->biter), addr + EDMA_TCD_BITER(ch)); - edma_writew(edma, le16_to_cpu(tcd->doff), addr + EDMA_TCD_DOFF(ch)); - - edma_writel(edma, le32_to_cpu(tcd->dlast_sga), addr + EDMA_TCD_DLAST_SGA(ch)); - - edma_writew(edma, le16_to_cpu(tcd->csr), addr + EDMA_TCD_CSR(ch)); -} - -static inline -void fsl_edma_fill_tcd(struct fsl_edma_hw_tcd *tcd, u32 src, u32 dst, - u16 attr, u16 soff, u32 nbytes, u32 slast, u16 citer, - u16 biter, u16 doff, u32 dlast_sga, bool major_int, - bool disable_req, bool enable_sg) -{ - u16 csr = 0; - - /* - * eDMA hardware SGs require the TCDs to be stored in little - * endian format irrespective of the register endian model. - * So we put the value in little endian in memory, waiting - * for fsl_edma_set_tcd_regs doing the swap. - */ - tcd->saddr = cpu_to_le32(src); - tcd->daddr = cpu_to_le32(dst); - - tcd->attr = cpu_to_le16(attr); - - tcd->soff = cpu_to_le16(EDMA_TCD_SOFF_SOFF(soff)); - - tcd->nbytes = cpu_to_le32(EDMA_TCD_NBYTES_NBYTES(nbytes)); - tcd->slast = cpu_to_le32(EDMA_TCD_SLAST_SLAST(slast)); - - tcd->citer = cpu_to_le16(EDMA_TCD_CITER_CITER(citer)); - tcd->doff = cpu_to_le16(EDMA_TCD_DOFF_DOFF(doff)); - - tcd->dlast_sga = cpu_to_le32(EDMA_TCD_DLAST_SGA_DLAST_SGA(dlast_sga)); - - tcd->biter = cpu_to_le16(EDMA_TCD_BITER_BITER(biter)); - if (major_int) - csr |= EDMA_TCD_CSR_INT_MAJOR; - - if (disable_req) - csr |= EDMA_TCD_CSR_D_REQ; - - if (enable_sg) - csr |= EDMA_TCD_CSR_E_SG; - - tcd->csr = cpu_to_le16(csr); -} - -static struct fsl_edma_desc *fsl_edma_alloc_desc(struct fsl_edma_chan *fsl_chan, - int sg_len) -{ - struct fsl_edma_desc *fsl_desc; - int i; - - fsl_desc = kzalloc(sizeof(*fsl_desc) + sizeof(struct fsl_edma_sw_tcd) * sg_len, - GFP_NOWAIT); - if (!fsl_desc) - return NULL; - - fsl_desc->echan = fsl_chan; - fsl_desc->n_tcds = sg_len; - for (i = 0; i < sg_len; i++) { - fsl_desc->tcd[i].vtcd = dma_pool_alloc(fsl_chan->tcd_pool, - GFP_NOWAIT, &fsl_desc->tcd[i].ptcd); - if (!fsl_desc->tcd[i].vtcd) - goto err; - } - return fsl_desc; - -err: - while (--i >= 0) - dma_pool_free(fsl_chan->tcd_pool, fsl_desc->tcd[i].vtcd, - fsl_desc->tcd[i].ptcd); - kfree(fsl_desc); - return NULL; -} - -static struct dma_async_tx_descriptor *fsl_edma_prep_dma_cyclic( - struct dma_chan *chan, dma_addr_t dma_addr, size_t buf_len, - size_t period_len, enum dma_transfer_direction direction, - unsigned long flags) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - struct fsl_edma_desc *fsl_desc; - dma_addr_t dma_buf_next; - int sg_len, i; - u32 src_addr, dst_addr, last_sg, nbytes; - u16 soff, doff, iter; - - if (!is_slave_direction(fsl_chan->fsc.dir)) - return NULL; - - sg_len = buf_len / period_len; - fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len); - if (!fsl_desc) - return NULL; - fsl_desc->iscyclic = true; - - dma_buf_next = dma_addr; - nbytes = fsl_chan->fsc.addr_width * fsl_chan->fsc.burst; - iter = period_len / nbytes; - - for (i = 0; i < sg_len; i++) { - if (dma_buf_next >= dma_addr + buf_len) - dma_buf_next = dma_addr; - - /* get next sg's physical address */ - last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd; - - if (fsl_chan->fsc.dir == DMA_MEM_TO_DEV) { - src_addr = dma_buf_next; - dst_addr = fsl_chan->fsc.dev_addr; - soff = fsl_chan->fsc.addr_width; - doff = 0; - } else { - src_addr = fsl_chan->fsc.dev_addr; - dst_addr = dma_buf_next; - soff = 0; - doff = fsl_chan->fsc.addr_width; - } - - fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, dst_addr, - fsl_chan->fsc.attr, soff, nbytes, 0, iter, - iter, doff, last_sg, true, false, true); - dma_buf_next += period_len; - } - - return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags); -} - -static struct dma_async_tx_descriptor *fsl_edma_prep_slave_sg( - struct dma_chan *chan, struct scatterlist *sgl, - unsigned int sg_len, enum dma_transfer_direction direction, - unsigned long flags, void *context) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - struct fsl_edma_desc *fsl_desc; - struct scatterlist *sg; - u32 src_addr, dst_addr, last_sg, nbytes; - u16 soff, doff, iter; - int i; - - if (!is_slave_direction(fsl_chan->fsc.dir)) - return NULL; - - fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len); - if (!fsl_desc) - return NULL; - fsl_desc->iscyclic = false; - - nbytes = fsl_chan->fsc.addr_width * fsl_chan->fsc.burst; - for_each_sg(sgl, sg, sg_len, i) { - /* get next sg's physical address */ - last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd; - - if (fsl_chan->fsc.dir == DMA_MEM_TO_DEV) { - src_addr = sg_dma_address(sg); - dst_addr = fsl_chan->fsc.dev_addr; - soff = fsl_chan->fsc.addr_width; - doff = 0; - } else { - src_addr = fsl_chan->fsc.dev_addr; - dst_addr = sg_dma_address(sg); - soff = 0; - doff = fsl_chan->fsc.addr_width; - } - - iter = sg_dma_len(sg) / nbytes; - if (i < sg_len - 1) { - last_sg = fsl_desc->tcd[(i + 1)].ptcd; - fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, - dst_addr, fsl_chan->fsc.attr, soff, - nbytes, 0, iter, iter, doff, last_sg, - false, false, true); - } else { - last_sg = 0; - fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, - dst_addr, fsl_chan->fsc.attr, soff, - nbytes, 0, iter, iter, doff, last_sg, - true, true, false); - } - } - - return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags); -} - -static void fsl_edma_xfer_desc(struct fsl_edma_chan *fsl_chan) -{ - struct virt_dma_desc *vdesc; - - vdesc = vchan_next_desc(&fsl_chan->vchan); - if (!vdesc) - return; - fsl_chan->edesc = to_fsl_edma_desc(vdesc); - fsl_edma_set_tcd_regs(fsl_chan, fsl_chan->edesc->tcd[0].vtcd); - fsl_edma_enable_request(fsl_chan); - fsl_chan->status = DMA_IN_PROGRESS; - fsl_chan->idle = false; -} - static irqreturn_t fsl_edma_tx_handler(int irq, void *dev_id) { struct fsl_edma_engine *fsl_edma = dev_id; unsigned int intr, ch; - void __iomem *base_addr; struct fsl_edma_chan *fsl_chan; + struct edma_regs *regs = &fsl_edma->regs; - base_addr = fsl_edma->membase; - - intr = edma_readl(fsl_edma, base_addr + EDMA_INTR); + intr = edma_readl(fsl_edma, regs->intl); if (!intr) return IRQ_NONE; for (ch = 0; ch < fsl_edma->n_chans; ch++) { if (intr & (0x1 << ch)) { - edma_writeb(fsl_edma, EDMA_CINT_CINT(ch), - base_addr + EDMA_CINT); + edma_writeb(fsl_edma, EDMA_CINT_CINT(ch), regs->cint); fsl_chan = &fsl_edma->chans[ch]; @@ -705,16 +76,16 @@ static irqreturn_t fsl_edma_err_handler(int irq, void *dev_id) { struct fsl_edma_engine *fsl_edma = dev_id; unsigned int err, ch; + struct edma_regs *regs = &fsl_edma->regs; - err = edma_readl(fsl_edma, fsl_edma->membase + EDMA_ERR); + err = edma_readl(fsl_edma, regs->errl); if (!err) return IRQ_NONE; for (ch = 0; ch < fsl_edma->n_chans; ch++) { if (err & (0x1 << ch)) { fsl_edma_disable_request(&fsl_edma->chans[ch]); - edma_writeb(fsl_edma, EDMA_CERR_CERR(ch), - fsl_edma->membase + EDMA_CERR); + edma_writeb(fsl_edma, EDMA_CERR_CERR(ch), regs->cerr); fsl_edma->chans[ch].status = DMA_ERROR; fsl_edma->chans[ch].idle = true; } @@ -730,25 +101,6 @@ static irqreturn_t fsl_edma_irq_handler(int irq, void *dev_id) return fsl_edma_err_handler(irq, dev_id); } -static void fsl_edma_issue_pending(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - - if (unlikely(fsl_chan->pm_state != RUNNING)) { - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - /* cannot submit due to suspend */ - return; - } - - if (vchan_issue_pending(&fsl_chan->vchan) && !fsl_chan->edesc) - fsl_edma_xfer_desc(fsl_chan); - - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); -} - static struct dma_chan *fsl_edma_xlate(struct of_phandle_args *dma_spec, struct of_dma *ofdma) { @@ -761,7 +113,8 @@ static struct dma_chan *fsl_edma_xlate(struct of_phandle_args *dma_spec, return NULL; mutex_lock(&fsl_edma->fsl_edma_mutex); - list_for_each_entry_safe(chan, _chan, &fsl_edma->dma_dev.channels, device_node) { + list_for_each_entry_safe(chan, _chan, + &fsl_edma->dma_dev.channels, device_node) { if (chan->client_count) continue; if ((chan->chan_id / chans_per_mux) == dma_spec->args[0]) { @@ -778,39 +131,12 @@ static struct dma_chan *fsl_edma_xlate(struct of_phandle_args *dma_spec, } } mutex_unlock(&fsl_edma->fsl_edma_mutex); - return NULL; -} -static int fsl_edma_alloc_chan_resources(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - - fsl_chan->tcd_pool = dma_pool_create("tcd_pool", chan->device->dev, - sizeof(struct fsl_edma_hw_tcd), - 32, 0); - return 0; -} - -static void fsl_edma_free_chan_resources(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - LIST_HEAD(head); - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - fsl_edma_disable_request(fsl_chan); - fsl_edma_chan_mux(fsl_chan, 0, false); - fsl_chan->edesc = NULL; - vchan_get_all_descriptors(&fsl_chan->vchan, &head); - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - - vchan_dma_desc_free_list(&fsl_chan->vchan, &head); - dma_pool_destroy(fsl_chan->tcd_pool); - fsl_chan->tcd_pool = NULL; + return NULL; } -static int -fsl_edma_irq_init(struct platform_device *pdev, struct fsl_edma_engine *fsl_edma) +static int fsl_edma_irq_init(struct platform_device *pdev, + struct fsl_edma_engine *fsl_edma) { int ret; @@ -827,21 +153,21 @@ fsl_edma_irq_init(struct platform_device *pdev, struct fsl_edma_engine *fsl_edma } if (fsl_edma->txirq == fsl_edma->errirq) { - ret = devm_request_irq(&pdev->dev, fsl_edma->txirq, + ret = request_irq(fsl_edma->txirq, fsl_edma_irq_handler, 0, "eDMA", fsl_edma); if (ret) { dev_err(&pdev->dev, "Can't register eDMA IRQ.\n"); return ret; } } else { - ret = devm_request_irq(&pdev->dev, fsl_edma->txirq, + ret = request_irq(fsl_edma->txirq, fsl_edma_tx_handler, 0, "eDMA tx", fsl_edma); if (ret) { dev_err(&pdev->dev, "Can't register eDMA tx IRQ.\n"); return ret; } - ret = devm_request_irq(&pdev->dev, fsl_edma->errirq, + ret = request_irq(fsl_edma->errirq, fsl_edma_err_handler, 0, "eDMA err", fsl_edma); if (ret) { dev_err(&pdev->dev, "Can't register eDMA err IRQ.\n"); @@ -852,14 +178,15 @@ fsl_edma_irq_init(struct platform_device *pdev, struct fsl_edma_engine *fsl_edma return 0; } + static void fsl_edma_irq_exit( struct platform_device *pdev, struct fsl_edma_engine *fsl_edma) { if (fsl_edma->txirq == fsl_edma->errirq) { - devm_free_irq(&pdev->dev, fsl_edma->txirq, fsl_edma); + free_irq(fsl_edma->txirq, fsl_edma); } else { - devm_free_irq(&pdev->dev, fsl_edma->txirq, fsl_edma); - devm_free_irq(&pdev->dev, fsl_edma->errirq, fsl_edma); + free_irq(fsl_edma->txirq, fsl_edma); + free_irq(fsl_edma->errirq, fsl_edma); } } @@ -876,6 +203,7 @@ static int fsl_edma_probe(struct platform_device *pdev) struct device_node *np = pdev->dev.of_node; struct fsl_edma_engine *fsl_edma; struct fsl_edma_chan *fsl_chan; + struct edma_regs *regs; struct resource *res; int len, chans; int ret, i; @@ -891,6 +219,8 @@ static int fsl_edma_probe(struct platform_device *pdev) if (!fsl_edma) return -ENOMEM; + fsl_edma->version = v1; + fsl_edma->n_chans = chans; mutex_init(&fsl_edma->fsl_edma_mutex); @@ -899,6 +229,9 @@ static int fsl_edma_probe(struct platform_device *pdev) if (IS_ERR(fsl_edma->membase)) return PTR_ERR(fsl_edma->membase); + fsl_edma_setup_regs(fsl_edma); + regs = &fsl_edma->regs; + for (i = 0; i < DMAMUX_NR; i++) { char clkname[32]; @@ -939,11 +272,11 @@ static int fsl_edma_probe(struct platform_device *pdev) fsl_chan->vchan.desc_free = fsl_edma_free_desc; vchan_init(&fsl_chan->vchan, &fsl_edma->dma_dev); - edma_writew(fsl_edma, 0x0, fsl_edma->membase + EDMA_TCD_CSR(i)); + edma_writew(fsl_edma, 0x0, ®s->tcd[i].csr); fsl_edma_chan_mux(fsl_chan, 0, false); } - edma_writel(fsl_edma, ~0, fsl_edma->membase + EDMA_INTR); + edma_writel(fsl_edma, ~0, regs->intl); ret = fsl_edma_irq_init(pdev, fsl_edma); if (ret) return ret; @@ -990,22 +323,11 @@ static int fsl_edma_probe(struct platform_device *pdev) } /* enable round robin arbitration */ - edma_writel(fsl_edma, EDMA_CR_ERGA | EDMA_CR_ERCA, fsl_edma->membase + EDMA_CR); + edma_writel(fsl_edma, EDMA_CR_ERGA | EDMA_CR_ERCA, regs->cr); return 0; } -static void fsl_edma_cleanup_vchan(struct dma_device *dmadev) -{ - struct fsl_edma_chan *chan, *_chan; - - list_for_each_entry_safe(chan, _chan, - &dmadev->channels, vchan.chan.device_node) { - list_del(&chan->vchan.chan.device_node); - tasklet_kill(&chan->vchan.task); - } -} - static int fsl_edma_remove(struct platform_device *pdev) { struct device_node *np = pdev->dev.of_node; @@ -1048,18 +370,19 @@ static int fsl_edma_resume_early(struct device *dev) { struct fsl_edma_engine *fsl_edma = dev_get_drvdata(dev); struct fsl_edma_chan *fsl_chan; + struct edma_regs *regs = &fsl_edma->regs; int i; for (i = 0; i < fsl_edma->n_chans; i++) { fsl_chan = &fsl_edma->chans[i]; fsl_chan->pm_state = RUNNING; - edma_writew(fsl_edma, 0x0, fsl_edma->membase + EDMA_TCD_CSR(i)); + edma_writew(fsl_edma, 0x0, ®s->tcd[i].csr); + if (fsl_chan->slave_id != 0) fsl_edma_chan_mux(fsl_chan, fsl_chan->slave_id, true); } - edma_writel(fsl_edma, EDMA_CR_ERGA | EDMA_CR_ERCA, - fsl_edma->membase + EDMA_CR); + edma_writel(fsl_edma, EDMA_CR_ERGA | EDMA_CR_ERCA, regs->cr); return 0; }