From patchwork Wed Mar 30 16:44:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Martin_Povi=C5=A1er?= X-Patchwork-Id: 12796105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BBD8BC433F5 for ; Wed, 30 Mar 2022 16:46:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=RxBzXZiIHJZAJm62L25Z/wmOcjCTuB4Dz5oXvNDA0/k=; b=ru6X3xRDDAxEoT zxGS2phi1amrPw6YXE/4HSj/gnhIdL9sOXr2qeulazFVcA07TBS0o+psmbO9gqujwvKa+Y/Zce5Rr puRQ7bgSrFH77VbzyRHa47cSRV8bl+ClZiG4v35zH/746anxnJmWV6+Xqyv/h7RZr2mg+1UzbtDAZ jjttNvRCX+EYgMeliDx1Cd+Eb5AtK6JtaTpHe3SjTjmkHzaXgS6da9G8aJ8ncfDMUYoNHBKvkAVeN W5ka5V31dMIP1ZZl8JpJBb6KAoFVoOeIm+1z51cEfmCbtpypsCMqzbzAr5xHK01EGdBy2LrCswHfs NKIEh/YxKqmZ891eUSLA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nZbRr-00Grnz-Q5; Wed, 30 Mar 2022 16:45:31 +0000 Received: from hutie.ust.cz ([185.8.165.127]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nZbRe-00Grks-Gu for linux-arm-kernel@lists.infradead.org; Wed, 30 Mar 2022 16:45:21 +0000 From: =?utf-8?q?Martin_Povi=C5=A1er?= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cutebit.org; s=mail; t=1648658714; bh=vezI7Vb6EsH9NmBmKIpXJGQJTm9trs6SWXfvudn0Ej0=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Be23g2jMXdEQrhnl5wsePDhbST24/waN01xf645qOcPOiEjqTY4uxfmnS0SIytQfA VbezQpWU9DmPlrrKyzT3tTEqLH/TyXYgZ6rIDyuzhdsxlzhRgbVOOIDSqyw6vRUwxk Arm6Y5IxWf5vA/FHHkdCxSgkvEC/lkY/dvlCt+7o= To: Hector Martin , Sven Peter , Vinod Koul , Rob Herring , Krzysztof Kozlowski Cc: Alyssa Rosenzweig , linux-arm-kernel@lists.infradead.org, dmaengine@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Mark Kettenis , =?utf-8?q?Martin_Povi=C5=A1er?= Subject: [PATCH 1/2] dt-bindings: dma: Add Apple ADMAC Date: Wed, 30 Mar 2022 18:44:57 +0200 Message-Id: <20220330164458.93055-2-povik+lin@cutebit.org> In-Reply-To: <20220330164458.93055-1-povik+lin@cutebit.org> References: <20220330164458.93055-1-povik+lin@cutebit.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220330_094519_229948_6B80A74E X-CRM114-Status: GOOD ( 11.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Apple's Audio DMA Controller (ADMAC) is used to fetch and store audio samples on Apple SoCs from the "Apple Silicon" family. Signed-off-by: Martin Povišer --- .../devicetree/bindings/dma/apple,admac.yaml | 73 +++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100644 Documentation/devicetree/bindings/dma/apple,admac.yaml diff --git a/Documentation/devicetree/bindings/dma/apple,admac.yaml b/Documentation/devicetree/bindings/dma/apple,admac.yaml new file mode 100644 index 000000000000..34f76a9a2983 --- /dev/null +++ b/Documentation/devicetree/bindings/dma/apple,admac.yaml @@ -0,0 +1,73 @@ +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/dma/apple,admac.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Apple Audio DMA Controller (ADMAC) + +description: | + Apple's Audio DMA Controller (ADMAC) is used to fetch and store + audio samples on Apple SoCs from the "Apple Silicon" family. + +maintainers: + - Martin Povišer + +allOf: + - $ref: "dma-controller.yaml#" + +properties: + compatible: + items: + - enum: + - apple,t6000-admac + - apple,t8103-admac + - const: apple,admac + + reg: + maxItems: 1 + + '#dma-cells': + const: 1 + + apple,internal-irq-destination: + $ref: /schemas/types.yaml#/definitions/uint32 + description: Index influencing internal routing of the IRQs + within the peripheral. + + interrupts: + maxItems: 1 + +required: + - compatible + - reg + - '#dma-cells' + - dma-channels + - apple,internal-irq-destination + - interrupts + +additionalProperties: false + +examples: + - | + #include + #include + + dart_sio: iommu@235004000 { + compatible = "apple,t8103-dart", "apple,dart"; + reg = <0x2 0x35004000 0x0 0x4000>; + interrupt-parent = <&aic>; + interrupts = ; + #iommu-cells = <1>; + }; + + admac: dma-controller@238200000 { + compatible = "apple,t8103-admac", "apple,admac"; + reg = <0x2 0x38200000 0x0 0x34000>; + dma-channels = <12>; + interrupt-parent = <&aic>; + interrupts = ; + #dma-cells = <1>; + iommus = <&dart_sio 2>; + apple,internal-irq-destination = <1>; + }; From patchwork Wed Mar 30 16:44:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Martin_Povi=C5=A1er?= X-Patchwork-Id: 12796106 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 17EC6C433EF for ; Wed, 30 Mar 2022 16:46:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=3DBQ17UFuZt0kP/9qj864x3QoVQUSqeRZX/REOSQBNU=; b=nmpcdwTSP4cWQm cpjyaLBqp/lJYWZI/hLIyFOaAX7HqCDzPYgnAqP8yysv6Ms0+54e4GQ5OdN9XOyeBUtQj4b4ftDL1 //JyBrm8McSBJDMd+8LsZo7lFwGnyi1UmheUzdUKP7D5ohOSO+IP/eDcJG5ANaBfY7BGcZdwGTVmy tYdFEhKvswROUT/x4ZWP/VrMs466t4SQbMRgyIMAQOwOJzl4/AZ1KV3ckvWAZsTS3eAygSku2dg9P YYnSf/BlDEHToYNEj+kXrRHyZN6IUxetaoLGjZ8UpsbU5jchPpCG6bVzsi2AIDScQK4uiCUYYBGx/ n9DOwwKpeZDk6t4aGh6Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nZbS1-00Grqs-Q3; Wed, 30 Mar 2022 16:45:41 +0000 Received: from hutie.ust.cz ([185.8.165.127]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nZbRe-00Grl9-Gx for linux-arm-kernel@lists.infradead.org; Wed, 30 Mar 2022 16:45:22 +0000 From: =?utf-8?q?Martin_Povi=C5=A1er?= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cutebit.org; s=mail; t=1648658716; bh=NmppNrMLK45qogHTSU7xLfTr2hjkNxVKbkHtWd69NIM=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=JtFs1Ns5Gl7njBF9q1xinSW7khmJT5pyuDP19ab4bpTdLfW4cc/n12i9Y/TfFZPpq Ie/80rMPV13HH3h/yMqvp90Vcbtd8hd4H6qrzi7T6iJ3+7GKiyhSKJ7EKLgU6ldUiD UF6SwcenCx9hAj9W/euUI8fQXFue2s/YO/vGt+fY= To: Hector Martin , Sven Peter , Vinod Koul , Rob Herring , Krzysztof Kozlowski Cc: Alyssa Rosenzweig , linux-arm-kernel@lists.infradead.org, dmaengine@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Mark Kettenis , =?utf-8?q?Martin_Povi=C5=A1er?= Subject: [PATCH 2/2] dmaengine: apple-admac: Add Apple ADMAC driver Date: Wed, 30 Mar 2022 18:44:58 +0200 Message-Id: <20220330164458.93055-3-povik+lin@cutebit.org> In-Reply-To: <20220330164458.93055-1-povik+lin@cutebit.org> References: <20220330164458.93055-1-povik+lin@cutebit.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220330_094519_257867_B3DEA157 X-CRM114-Status: GOOD ( 24.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add driver for Audio DMA Controller present on Apple SoCs from the "Apple Silicon" family. Signed-off-by: Martin Povišer --- MAINTAINERS | 2 + drivers/dma/Kconfig | 8 + drivers/dma/Makefile | 1 + drivers/dma/apple-admac.c | 799 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 810 insertions(+) create mode 100644 drivers/dma/apple-admac.c diff --git a/MAINTAINERS b/MAINTAINERS index d9812b70c322..c6311a921190 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1837,6 +1837,7 @@ T: git https://github.com/AsahiLinux/linux.git F: Documentation/devicetree/bindings/arm/apple.yaml F: Documentation/devicetree/bindings/arm/apple/* F: Documentation/devicetree/bindings/clock/apple,nco.yaml +F: Documentation/devicetree/bindings/dma/apple,admac.yaml F: Documentation/devicetree/bindings/i2c/apple,i2c.yaml F: Documentation/devicetree/bindings/interrupt-controller/apple,* F: Documentation/devicetree/bindings/mailbox/apple,mailbox.yaml @@ -1846,6 +1847,7 @@ F: Documentation/devicetree/bindings/power/apple* F: Documentation/devicetree/bindings/watchdog/apple,wdt.yaml F: arch/arm64/boot/dts/apple/ F: drivers/clk/clk-apple-nco.c +F: drivers/dma/apple-admac.c F: drivers/i2c/busses/i2c-pasemi-core.c F: drivers/i2c/busses/i2c-pasemi-platform.c F: drivers/irqchip/irq-apple-aic.c diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig index d5de3f77d3aa..dd13bc47eb06 100644 --- a/drivers/dma/Kconfig +++ b/drivers/dma/Kconfig @@ -85,6 +85,14 @@ config AMCC_PPC440SPE_ADMA help Enable support for the AMCC PPC440SPe RAID engines. +config APPLE_ADMAC + tristate "Apple ADMAC support" + depends on ARCH_APPLE || COMPILE_TEST + select DMA_ENGINE + default ARCH_APPLE + help + Enable support for Audio DMA Controller found on Apple Silicon SoCs. + config AT_HDMAC tristate "Atmel AHB DMA support" depends on ARCH_AT91 diff --git a/drivers/dma/Makefile b/drivers/dma/Makefile index 616d926cf2a5..28e743c7d86f 100644 --- a/drivers/dma/Makefile +++ b/drivers/dma/Makefile @@ -17,6 +17,7 @@ obj-$(CONFIG_ALTERA_MSGDMA) += altera-msgdma.o obj-$(CONFIG_AMBA_PL08X) += amba-pl08x.o obj-$(CONFIG_AMCC_PPC440SPE_ADMA) += ppc4xx/ obj-$(CONFIG_AMD_PTDMA) += ptdma/ +obj-$(CONFIG_APPLE_ADMAC) += apple-admac.o obj-$(CONFIG_AT_HDMAC) += at_hdmac.o obj-$(CONFIG_AT_XDMAC) += at_xdmac.o obj-$(CONFIG_AXI_DMAC) += dma-axi-dmac.o diff --git a/drivers/dma/apple-admac.c b/drivers/dma/apple-admac.c new file mode 100644 index 000000000000..e7fd9856474e --- /dev/null +++ b/drivers/dma/apple-admac.c @@ -0,0 +1,799 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Driver for Audio DMA Controller (ADMAC) on t8103 (M1) and other Apple chips + * + * Copyright (C) The Asahi Linux Contributors + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "dmaengine.h" + +#define NCHANNELS_MAX 64 +#define IRQ_NINDICES 4 + +#define RING_WRITE_SLOT GENMASK(1, 0) +#define RING_READ_SLOT GENMASK(5, 4) +#define RING_FULL BIT(9) +#define RING_EMPTY BIT(8) +#define RING_ERR BIT(10) + +#define STATUS_DESC_DONE BIT(0) +#define STATUS_ERR BIT(6) + +#define FLAG_DESC_NOTIFY BIT(16) + +#define REG_TX_START 0x0000 +#define REG_TX_STOP 0x0004 +#define REG_RX_START 0x0008 +#define REG_RX_STOP 0x000c + +#define REG_CHAN_CTL(ch) (0x8000 + (ch) * 0x200) +#define REG_CHAN_CTL_RST_RINGS BIT(0) + +#define REG_DESC_RING(ch) (0x8070 + (ch) * 0x200) +#define REG_REPORT_RING(ch) (0x8074 + (ch) * 0x200) + +#define REG_RESIDUE(ch) (0x8064 + (ch) * 0x200) + +#define REG_BUS_WIDTH(ch) (0x8040 + (ch) * 0x200) + +#define BUS_WIDTH_8BIT 0x00 +#define BUS_WIDTH_16BIT 0x01 +#define BUS_WIDTH_32BIT 0x02 +#define BUS_WIDTH_FRAME_2_WORDS 0x10 +#define BUS_WIDTH_FRAME_4_WORDS 0x20 + +#define CHAN_BUFSIZE 0x8000 + +#define REG_CHAN_FIFOCTL(ch) (0x8054 + (ch) * 0x200) +#define CHAN_FIFOCTL_LIMIT GENMASK(31, 16) +#define CHAN_FIFOCTL_THRESHOLD GENMASK(15, 0) + +#define REG_DESC_WRITE(ch) (0x10000 + ((ch) / 2) * 0x4 + ((ch) & 1) * 0x4000) +#define REG_REPORT_READ(ch) (0x10100 + ((ch) / 2) * 0x4 + ((ch) & 1) * 0x4000) + +#define REG_TX_INTSTATE(idx) (0x0030 + (idx) * 4) +#define REG_RX_INTSTATE(idx) (0x0040 + (idx) * 4) +#define REG_CHAN_INTSTATUS(ch, idx) (0x8010 + (ch) * 0x200 + (idx) * 4) +#define REG_CHAN_INTMASK(ch, idx) (0x8020 + (ch) * 0x200 + (idx) * 4) + +struct admac_data; +struct admac_tx; + +struct admac_chan { + int no; + struct admac_data *host; + struct dma_chan chan; + struct tasklet_struct tasklet; + + spinlock_t lock; + struct admac_tx *current_tx; + int nperiod_acks; + + struct list_head submitted; + struct list_head issued; +}; + +struct admac_data { + struct dma_device dma; + struct device *dev; + __iomem void *base; + + int irq_index; + int nchannels; + struct admac_chan channels[]; +}; + +struct admac_tx { + struct dma_async_tx_descriptor tx; + bool cyclic; + dma_addr_t buf_addr; + dma_addr_t buf_end; + size_t buf_len; + size_t period_len; + + size_t submitted_pos; + size_t reclaimed_pos; + + struct list_head node; +}; + +static void admac_poke(struct admac_data *ad, int reg, u32 val) +{ + writel_relaxed(val, ad->base + reg); +} + +static u32 admac_peek(struct admac_data *ad, int reg) +{ + return readl_relaxed(ad->base + reg); +} + +static void admac_modify(struct admac_data *ad, int reg, u32 mask, u32 val) +{ + void __iomem *addr = ad->base + reg; + u32 curr = readl_relaxed(addr); + + writel_relaxed((curr & ~mask) | (val & mask), addr); +} + +static struct admac_chan *to_admac_chan(struct dma_chan *chan) +{ + return container_of(chan, struct admac_chan, chan); +} + +static struct admac_tx *to_admac_tx(struct dma_async_tx_descriptor *tx) +{ + return container_of(tx, struct admac_tx, tx); +} + +static enum dma_transfer_direction admac_chan_direction(int channo) +{ + return (channo & 1) ? DMA_DEV_TO_MEM : DMA_MEM_TO_DEV; +} + +static dma_cookie_t admac_tx_submit(struct dma_async_tx_descriptor *tx) +{ + struct admac_tx *adtx = to_admac_tx(tx); + struct admac_chan *adchan = to_admac_chan(tx->chan); + unsigned long flags; + dma_cookie_t cookie; + + spin_lock_irqsave(&adchan->lock, flags); + cookie = dma_cookie_assign(tx); + list_add_tail(&adtx->node, &adchan->submitted); + spin_unlock_irqrestore(&adchan->lock, flags); + + return cookie; +} + +static int admac_desc_free(struct dma_async_tx_descriptor *tx) +{ + struct admac_tx *adtx = to_admac_tx(tx); + + devm_kfree(to_admac_chan(tx->chan)->host->dev, adtx); + return 0; +} + +static struct dma_async_tx_descriptor *admac_prep_dma_cyclic( + struct dma_chan *chan, dma_addr_t buf_addr, size_t buf_len, + size_t period_len, enum dma_transfer_direction direction, + unsigned long flags) +{ + struct admac_chan *adchan = container_of(chan, struct admac_chan, chan); + struct admac_tx *adtx; + + if (direction != admac_chan_direction(adchan->no)) + return NULL; + + adtx = devm_kzalloc(adchan->host->dev, sizeof(*adtx), GFP_NOWAIT); + if (!adtx) + return NULL; + + adtx->cyclic = true; + + adtx->buf_addr = buf_addr; + adtx->buf_len = buf_len; + adtx->buf_end = buf_addr + buf_len; + adtx->period_len = period_len; + + adtx->submitted_pos = 0; + adtx->reclaimed_pos = 0; + + dma_async_tx_descriptor_init(&adtx->tx, chan); + adtx->tx.tx_submit = admac_tx_submit; + adtx->tx.desc_free = admac_desc_free; + + return &adtx->tx; +} + +/* + * Write one hardware descriptor for a dmaengine cyclic transaction. + */ +static void admac_cyclic_write_one_desc(struct admac_data *ad, int channo, + struct admac_tx *tx) +{ + dma_addr_t addr; + + if (WARN_ON(!tx->cyclic)) + return; + + addr = tx->buf_addr + (tx->submitted_pos % tx->buf_len); + WARN_ON(addr + tx->period_len > tx->buf_end); + + dev_dbg(ad->dev, "ch%d descriptor: addr=0x%pad len=0x%zx flags=0x%x\n", + channo, addr, tx->period_len, FLAG_DESC_NOTIFY); + + admac_poke(ad, REG_DESC_WRITE(channo), addr); + admac_poke(ad, REG_DESC_WRITE(channo), addr >> 32); + admac_poke(ad, REG_DESC_WRITE(channo), tx->period_len); + admac_poke(ad, REG_DESC_WRITE(channo), FLAG_DESC_NOTIFY); + + tx->submitted_pos += tx->period_len; + tx->submitted_pos %= 2 * tx->buf_len; +} + +/* + * Write all the hardware descriptors for a cyclic transaction + * there is space for. + */ +static void admac_cyclic_write_desc(struct admac_data *ad, int channo, + struct admac_tx *tx) +{ + int i; + + for (i = 0; i < 4; i++) { + if (admac_peek(ad, REG_DESC_RING(channo)) & RING_FULL) + break; + admac_cyclic_write_one_desc(ad, channo, tx); + } +} + +static int admac_ring_noccupied_slots(int ringval) +{ + int wrslot = FIELD_GET(RING_WRITE_SLOT, ringval); + int rdslot = FIELD_GET(RING_READ_SLOT, ringval); + + if (wrslot != rdslot) { + return (wrslot + 4 - rdslot) % 4; + } else { + WARN_ON((ringval & (RING_FULL | RING_EMPTY)) == 0); + + if (ringval & RING_FULL) + return 4; + else + return 0; + } +} + +/* + * Read from hardware the residue of a cyclic dmaengine transaction. + */ +static u32 admac_cyclic_read_residue(struct admac_data *ad, int channo, + struct admac_tx *adtx) +{ + u32 ring1, ring2; + u32 residue1, residue2; + int nreports; + size_t pos; + + ring1 = admac_peek(ad, REG_REPORT_RING(channo)); + residue1 = admac_peek(ad, REG_RESIDUE(channo)); + ring2 = admac_peek(ad, REG_REPORT_RING(channo)); + residue2 = admac_peek(ad, REG_RESIDUE(channo)); + + if (residue2 > residue1) { + /* + * Controller must have loaded next descriptor between + * the two residue reads + */ + nreports = admac_ring_noccupied_slots(ring1) + 1; + } else { + /* No descriptor load between the two reads, ring2 is safe to use */ + nreports = admac_ring_noccupied_slots(ring2); + } + + pos = adtx->reclaimed_pos + adtx->period_len * (nreports + 1) - residue2; + + return adtx->buf_len - pos % adtx->buf_len; +} + +static enum dma_status admac_tx_status(struct dma_chan *chan, dma_cookie_t cookie, + struct dma_tx_state *txstate) +{ + struct admac_chan *adchan = to_admac_chan(chan); + struct admac_data *ad = adchan->host; + struct admac_tx *adtx; + + enum dma_status ret; + size_t residue; + unsigned long flags; + + ret = dma_cookie_status(chan, cookie, txstate); + if (ret == DMA_COMPLETE || !txstate) + return ret; + + spin_lock_irqsave(&adchan->lock, flags); + adtx = adchan->current_tx; + + if (adtx && adtx->tx.cookie == cookie) { + ret = DMA_IN_PROGRESS; + residue = admac_cyclic_read_residue(ad, adchan->no, adtx); + } else { + ret = DMA_IN_PROGRESS; + residue = 0; + list_for_each_entry(adtx, &adchan->issued, node) { + if (adtx->tx.cookie == cookie) { + residue = adtx->buf_len; + break; + } + } + } + spin_unlock_irqrestore(&adchan->lock, flags); + + dma_set_residue(txstate, residue); + return ret; +} + +static void admac_start_chan(struct admac_chan *adchan) +{ + u32 startbit = 1 << (adchan->no / 2); + + switch (admac_chan_direction(adchan->no)) { + case DMA_MEM_TO_DEV: + admac_poke(adchan->host, REG_TX_START, startbit); + break; + case DMA_DEV_TO_MEM: + admac_poke(adchan->host, REG_RX_START, startbit); + break; + default: + break; + } + dev_dbg(adchan->host->dev, "ch%d start\n", adchan->no); +} + +static void admac_stop_chan(struct admac_chan *adchan) +{ + u32 stopbit = 1 << (adchan->no / 2); + + switch (admac_chan_direction(adchan->no)) { + case DMA_MEM_TO_DEV: + admac_poke(adchan->host, REG_TX_STOP, stopbit); + break; + case DMA_DEV_TO_MEM: + admac_poke(adchan->host, REG_RX_STOP, stopbit); + break; + default: + break; + } + dev_dbg(adchan->host->dev, "ch%d stop\n", adchan->no); +} + +static void admac_start_current_tx(struct admac_chan *adchan) +{ + struct admac_data *ad = adchan->host; + int ch = adchan->no; + + admac_poke(ad, REG_CHAN_INTSTATUS(ch, ad->irq_index), + STATUS_DESC_DONE | STATUS_ERR); + admac_poke(ad, REG_CHAN_CTL(ch), REG_CHAN_CTL_RST_RINGS); + admac_poke(ad, REG_CHAN_CTL(ch), 0); + + admac_cyclic_write_one_desc(ad, ch, adchan->current_tx); + admac_start_chan(adchan); + admac_cyclic_write_desc(ad, ch, adchan->current_tx); +} + +static void admac_issue_pending(struct dma_chan *chan) +{ + struct admac_chan *adchan = to_admac_chan(chan); + struct admac_tx *tx; + unsigned long flags; + + spin_lock_irqsave(&adchan->lock, flags); + list_splice_tail_init(&adchan->submitted, &adchan->issued); + if (!list_empty(&adchan->issued) && !adchan->current_tx) { + tx = list_first_entry(&adchan->issued, struct admac_tx, node); + list_del(&tx->node); + + adchan->current_tx = tx; + adchan->nperiod_acks = 0; + admac_start_current_tx(adchan); + } + spin_unlock_irqrestore(&adchan->lock, flags); +} + +static int admac_pause(struct dma_chan *chan) +{ + struct admac_chan *adchan = to_admac_chan(chan); + + admac_stop_chan(adchan); + + return 0; +} + +static int admac_resume(struct dma_chan *chan) +{ + struct admac_chan *adchan = to_admac_chan(chan); + + admac_start_chan(adchan); + + return 0; +} + +static int admac_terminate_all(struct dma_chan *chan) +{ + struct admac_chan *adchan = to_admac_chan(chan); + struct admac_tx *adtx, *_adtx; + unsigned long flags; + LIST_HEAD(head); + + spin_lock_irqsave(&adchan->lock, flags); + admac_stop_chan(adchan); + adchan->current_tx = NULL; + list_splice_tail_init(&adchan->submitted, &head); + list_splice_tail_init(&adchan->issued, &head); + spin_unlock_irqrestore(&adchan->lock, flags); + + list_for_each_entry_safe(adtx, _adtx, &head, node) { + list_del(&adtx->node); + admac_desc_free(&adtx->tx); + } + + return 0; +} + +static int admac_alloc_chan_resources(struct dma_chan *chan) +{ + struct admac_chan *adchan = to_admac_chan(chan); + struct admac_data *ad = adchan->host; + + dma_cookie_init(&adchan->chan); + admac_poke(ad, REG_CHAN_INTSTATUS(adchan->no, ad->irq_index), + STATUS_DESC_DONE | STATUS_ERR); + admac_poke(ad, REG_CHAN_INTMASK(adchan->no, ad->irq_index), + STATUS_DESC_DONE | STATUS_ERR); + return 0; +} + +static void admac_free_chan_resources(struct dma_chan *chan) +{ + struct admac_chan *adchan = to_admac_chan(chan); + + admac_terminate_all(chan); + tasklet_kill(&adchan->tasklet); +} + +static struct dma_chan *admac_dma_of_xlate(struct of_phandle_args *dma_spec, + struct of_dma *ofdma) +{ + struct admac_data *ad = (struct admac_data *) ofdma->of_dma_data; + unsigned int index; + + if (dma_spec->args_count != 1) + return NULL; + + index = dma_spec->args[0]; + + if (index >= ad->nchannels) { + dev_err(ad->dev, "channel index %u out of bounds\n", index); + return NULL; + } + + return &ad->channels[index].chan; +} + +static int admac_drain_reports(struct admac_data *ad, int channo) +{ + int count; + + for (count = 0; count < 4; count++) { + u32 countval_hi, countval_lo, unk1, flags; + + if (admac_peek(ad, REG_REPORT_RING(channo)) & RING_EMPTY) + break; + + countval_lo = admac_peek(ad, REG_REPORT_READ(channo)); + countval_hi = admac_peek(ad, REG_REPORT_READ(channo)); + unk1 = admac_peek(ad, REG_REPORT_READ(channo)); + flags = admac_peek(ad, REG_REPORT_READ(channo)); + + dev_dbg(ad->dev, "ch%d report: countval=0x%llx unk1=0x%x flags=0x%x\n", + channo, ((u64) countval_hi) << 32 | countval_lo, unk1, flags); + } + + return count; +} + +static void admac_handle_status_err(struct admac_data *ad, int channo) +{ + bool handled = false; + + if (admac_peek(ad, REG_DESC_RING(channo) & RING_ERR)) { + admac_poke(ad, REG_DESC_RING(channo), RING_ERR); + dev_err(ad->dev, "ch%d descriptor ring error\n", channo); + handled = true; + } + + if (admac_peek(ad, REG_REPORT_RING(channo)) & RING_ERR) { + admac_poke(ad, REG_REPORT_RING(channo), RING_ERR); + dev_err(ad->dev, "ch%d report ring error\n", channo); + handled = true; + } + + if (unlikely(!handled)) { + dev_err(ad->dev, "ch%d unknown error, masking errors as cause of IRQs\n", channo); + admac_modify(ad, REG_CHAN_INTMASK(channo, ad->irq_index), + STATUS_ERR, 0); + } +} + +static void admac_handle_status_desc_done(struct admac_data *ad, int channo) +{ + struct admac_chan *adchan = &ad->channels[channo]; + unsigned long flags; + int nreports; + + admac_poke(ad, REG_CHAN_INTSTATUS(channo, ad->irq_index), + STATUS_DESC_DONE); + + spin_lock_irqsave(&adchan->lock, flags); + nreports = admac_drain_reports(ad, channo); + + if (adchan->current_tx) { + struct admac_tx *tx = adchan->current_tx; + + adchan->nperiod_acks += nreports; + tx->reclaimed_pos += nreports * tx->period_len; + tx->reclaimed_pos %= 2 * tx->buf_len; + + admac_cyclic_write_desc(ad, channo, tx); + tasklet_schedule(&adchan->tasklet); + } + spin_unlock_irqrestore(&adchan->lock, flags); +} + +static void admac_handle_chan_int(struct admac_data *ad, int no) +{ + u32 cause = admac_peek(ad, REG_CHAN_INTSTATUS(no, ad->irq_index)); + + if (cause & STATUS_ERR) + admac_handle_status_err(ad, no); + + if (cause & STATUS_DESC_DONE) + admac_handle_status_desc_done(ad, no); +} + +static irqreturn_t admac_interrupt(int irq, void *devid) +{ + struct admac_data *ad = devid; + u32 rx_intstate, tx_intstate; + int i; + + rx_intstate = admac_peek(ad, REG_RX_INTSTATE(ad->irq_index)); + tx_intstate = admac_peek(ad, REG_TX_INTSTATE(ad->irq_index)); + + if (!tx_intstate && !rx_intstate) + return IRQ_NONE; + + for (i = 0; i < ad->nchannels; i += 2) { + if (tx_intstate & 1) + admac_handle_chan_int(ad, i); + tx_intstate >>= 1; + } + + for (i = 1; i < ad->nchannels; i += 2) { + if (rx_intstate & 1) + admac_handle_chan_int(ad, i); + rx_intstate >>= 1; + } + + return IRQ_HANDLED; +} + +static void admac_chan_tasklet(struct tasklet_struct *t) +{ + struct admac_chan *adchan = from_tasklet(adchan, t, tasklet); + struct admac_tx *adtx; + struct dmaengine_desc_callback cb; + struct dmaengine_result tx_result; + int nacks; + + spin_lock_irq(&adchan->lock); + adtx = adchan->current_tx; + nacks = adchan->nperiod_acks; + adchan->nperiod_acks = 0; + spin_unlock_irq(&adchan->lock); + + if (!adtx || !nacks) + return; + + tx_result.result = DMA_TRANS_NOERROR; + tx_result.residue = 0; + + dmaengine_desc_get_callback(&adtx->tx, &cb); + while (nacks--) + dmaengine_desc_callback_invoke(&cb, &tx_result); +} + +static int admac_device_config(struct dma_chan *chan, + struct dma_slave_config *config) +{ + struct admac_chan *adchan = to_admac_chan(chan); + struct admac_data *ad = adchan->host; + bool is_tx = admac_chan_direction(adchan->no) == DMA_MEM_TO_DEV; + int wordsize = 0; + u32 bus_width = 0; + + switch (is_tx ? config->dst_addr_width : config->src_addr_width) { + case DMA_SLAVE_BUSWIDTH_1_BYTE: + wordsize = 1; + bus_width |= BUS_WIDTH_8BIT; + break; + case DMA_SLAVE_BUSWIDTH_2_BYTES: + wordsize = 2; + bus_width |= BUS_WIDTH_16BIT; + break; + case DMA_SLAVE_BUSWIDTH_4_BYTES: + wordsize = 4; + bus_width |= BUS_WIDTH_32BIT; + break; + default: + return -EINVAL; + } + + /* + * We take port_window_size to be the number of words in a frame. + * + * The controller has some means of out-of-band signalling, to the peripheral, + * of words position in a frame. That's where the importance of this control + * comes from. + */ + switch (is_tx ? config->dst_port_window_size : config->src_port_window_size) { + case 0 ... 1: + break; + case 2: + bus_width |= BUS_WIDTH_FRAME_2_WORDS; + break; + case 4: + bus_width |= BUS_WIDTH_FRAME_4_WORDS; + break; + default: + return -EINVAL; + } + + admac_poke(ad, REG_BUS_WIDTH(adchan->no), bus_width); + + /* + * By FIFOCTL_LIMIT we seem to set the maximal number of bytes allowed to be + * held in controller's per-channel FIFO. Transfers seem to be triggered + * around the time FIFO occupancy touches FIFOCTL_THRESHOLD. + * + * The numbers we set are more or less arbitrary. + */ + admac_poke(ad, REG_CHAN_FIFOCTL(adchan->no), + FIELD_PREP(CHAN_FIFOCTL_LIMIT, 0x30 * wordsize) + | FIELD_PREP(CHAN_FIFOCTL_THRESHOLD, 0x18 * wordsize)); + + return 0; +} + +static int admac_probe(struct platform_device *pdev) +{ + struct device_node *np = pdev->dev.of_node; + struct admac_data *ad; + struct dma_device *dma; + int nchannels; + int err, irq, i; + + err = of_property_read_u32(np, "dma-channels", &nchannels); + if (err || nchannels > NCHANNELS_MAX) { + dev_err(&pdev->dev, "missing or invalid dma-channels property\n"); + return -EINVAL; + } + + ad = devm_kzalloc(&pdev->dev, struct_size(ad, channels, nchannels), GFP_KERNEL); + if (!ad) + return -ENOMEM; + + platform_set_drvdata(pdev, ad); + ad->dev = &pdev->dev; + ad->nchannels = nchannels; + + err = of_property_read_u32(np, "apple,internal-irq-destination", + &ad->irq_index); + if (err || ad->irq_index >= IRQ_NINDICES) { + dev_err(&pdev->dev, "missing or invalid apple,internal-irq-destination property\n"); + return -EINVAL; + } + + irq = platform_get_irq(pdev, 0); + if (irq < 0) { + dev_err(&pdev->dev, "unable to obtain interrupt resource\n"); + return irq; + } + + err = devm_request_irq(&pdev->dev, irq, admac_interrupt, + 0, dev_name(&pdev->dev), ad); + if (err) { + dev_err(&pdev->dev, "unable to register interrupt: %d\n", err); + return err; + } + + ad->base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(ad->base)) { + dev_err(&pdev->dev, "unable to obtain MMIO resource\n"); + return PTR_ERR(ad->base); + } + + dma = &ad->dma; + + dma_cap_set(DMA_PRIVATE, dma->cap_mask); + dma_cap_set(DMA_CYCLIC, dma->cap_mask); + + dma->dev = &pdev->dev; + dma->device_alloc_chan_resources = admac_alloc_chan_resources; + dma->device_free_chan_resources = admac_free_chan_resources; + dma->device_tx_status = admac_tx_status; + dma->device_issue_pending = admac_issue_pending; + dma->device_terminate_all = admac_terminate_all; + dma->device_prep_dma_cyclic = admac_prep_dma_cyclic; + dma->device_config = admac_device_config; + dma->device_pause = admac_pause; + dma->device_resume = admac_resume; + + dma->directions = BIT(DMA_MEM_TO_DEV) | BIT(DMA_DEV_TO_MEM); + dma->residue_granularity = DMA_RESIDUE_GRANULARITY_BURST; + dma->dst_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) | + BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) | + BIT(DMA_SLAVE_BUSWIDTH_4_BYTES); + + INIT_LIST_HEAD(&dma->channels); + for (i = 0; i < nchannels; i++) { + struct admac_chan *adchan = &ad->channels[i]; + + adchan->host = ad; + adchan->no = i; + adchan->chan.device = &ad->dma; + spin_lock_init(&adchan->lock); + INIT_LIST_HEAD(&adchan->submitted); + INIT_LIST_HEAD(&adchan->issued); + list_add_tail(&adchan->chan.device_node, &dma->channels); + tasklet_setup(&adchan->tasklet, admac_chan_tasklet); + } + + err = dma_async_device_register(&ad->dma); + if (err) { + dev_err(&pdev->dev, "failed to register DMA device: %d\n", err); + return err; + } + + err = of_dma_controller_register(pdev->dev.of_node, admac_dma_of_xlate, ad); + if (err) { + dev_err(&pdev->dev, "failed to register with OF: %d\n", err); + dma_async_device_unregister(&ad->dma); + return err; + } + + dev_dbg(&pdev->dev, "all good, ready to go!\n"); + + return 0; +} + +static int admac_remove(struct platform_device *pdev) +{ + struct admac_data *ad = platform_get_drvdata(pdev); + + of_dma_controller_free(pdev->dev.of_node); + dma_async_device_unregister(&ad->dma); + + return 0; +} + +static const struct of_device_id admac_of_match[] = { + { .compatible = "apple,admac", }, + { } +}; +MODULE_DEVICE_TABLE(of, admac_of_match); + +static struct platform_driver apple_admac_driver = { + .driver = { + .name = "apple-admac", + .of_match_table = admac_of_match, + }, + .probe = admac_probe, + .remove = admac_remove, +}; +module_platform_driver(apple_admac_driver); + +MODULE_AUTHOR("Martin Povišer "); +MODULE_DESCRIPTION("Driver for Audio DMA Controller (ADMAC) on Apple SoCs"); +MODULE_LICENSE("GPL");