From patchwork Wed Mar 15 19:34:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiner Kallweit X-Patchwork-Id: 9626539 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id CF7D560244 for ; Wed, 15 Mar 2017 19:46:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B27CF2865C for ; Wed, 15 Mar 2017 19:46:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A4F5628672; Wed, 15 Mar 2017 19:46:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, DKIM_VALID, FREEMAIL_FROM autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 135A72865C for ; Wed, 15 Mar 2017 19:46:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date: Message-ID:References:To:Subject:From:Reply-To:Content-ID:Content-Description :Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=7F/v/NSd8hzn1LzTrk56st6KlhztJLJyGTGJB1D2wUQ=; b=njlVSnm6SCnWDf irbPZXl/66f96GJv0NpDP25WGSykQpvx/lhF1pMK5RzcDuf5iDOIaHY0wEInj/UskGGS9KXdpLWLb PLuphjqE+Z2gFV3mr9XkFe92bh+xJb9k7XMFy/Fwdk3+JYvwh/rGBekderGvB7c/ifYxb7KUaArYz 6PDIxaqIDbFOobVTmE+wr6j3c/MbiR2SN2+n1YXvQ+P/1C+jzx0Er1kjMKT+hJO8FOdWm8erf0j0N UwLG+yi8F5vJqvRxB2S+/dv9WzONnpz94/Q0ubLiq1jrzyXq1XWnpbSvSOmdftTE3cwFua8jTJ/cN LdV+SpnDTYQ7KRU6N7RA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1coEsT-00032p-JD; Wed, 15 Mar 2017 19:46:33 +0000 Received: from mail-wm0-x244.google.com ([2a00:1450:400c:c09::244]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1coEhf-0003uG-Fo for linux-amlogic@lists.infradead.org; Wed, 15 Mar 2017 19:35:27 +0000 Received: by mail-wm0-x244.google.com with SMTP id z63so6331704wmg.2 for ; Wed, 15 Mar 2017 12:35:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:subject:to:references:cc:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=jpcOZsjhO3Us2QuKvZtD8CPawvh3XzUJak6B6zei518=; b=u1dRHWsUS7W5HZbxhXB7N0zA+rHgA/tAQtqqHjIToksZEe2YqzLIkITbi/vWktd6z6 cjCVNQLmvqnvSiytgfae7BD76eT9unmdJ9iYaVNpc4H/PjrumwAODevV5vC3QLQtKOpw ox7YWTPprQXY720G5D4YGrULZSN/0ksvVHqiHKs8OKKEWR6CJJ96As3yN1TykY1TPHsq An9W+exfetwXCxtkNHxnXBd1+BN6GsMflzlvrha8ojstJWeI3sWlX5Z89k0wlE5eDyT4 D+7G0J5ZuxUQm6kWg3lfuKrKFW2R1YA1Q/VyGa8hHEcRV7par6T+/lhEnYWuPky6KeGZ CEng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:references:cc:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=jpcOZsjhO3Us2QuKvZtD8CPawvh3XzUJak6B6zei518=; b=twmXbULVtiwV/m/Mbvw2R9yqkqkaHMcJXY5OktflGTuZ6uQyIr6l08fNhR8YSmmVTR R70UHdGRx/tE+IJUaJwKpFC9a3rmewXxj91YuSP5HRDhkCMW72acnYdqxq2YFcK4/y9t 57YgTLXztEUXrfIV075bCQiolP6+4DyAfDY6wyTnNtbCe+DxyBFJLhn3baPa2YElmZrB g+WmaMmQaF52zprNMeEEGK0UCZVNUsU2E8F29rEjWbc5MYUxWElNLxD7N20t2QeHWBqA QD+UG1I3pfyYMKOD8PGccbfRalrDIf9q2N+YVcQu/400SSH6US8xkput6LpELXuFMaQh qCUQ== X-Gm-Message-State: AFeK/H0eF+EeNZkT5rFJ6Txbs23ndqrH9yWPOgWN1lCX6kmcJY8OhLxmQPvg9ZLdBc8OqQ== X-Received: by 10.28.152.11 with SMTP id a11mr59560wme.64.1489606502053; Wed, 15 Mar 2017 12:35:02 -0700 (PDT) Received: from ?IPv6:2003:c6:ebdc:4000:c5e4:fad9:4bd9:799d? (p200300C6EBDC4000C5E4FAD94BD9799D.dip0.t-ipconnect.de. [2003:c6:ebdc:4000:c5e4:fad9:4bd9:799d]) by smtp.googlemail.com with ESMTPSA id t194sm1587104wmd.13.2017.03.15.12.35.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 15 Mar 2017 12:35:01 -0700 (PDT) From: Heiner Kallweit Subject: [PATCH 5/5] mmc: meson-gx: switch to descriptor chain mode To: Ulf Hansson , Kevin Hilman References: Message-ID: <57af6536-edf9-a2aa-40fe-a90338245d0e@gmail.com> Date: Wed, 15 Mar 2017 20:34:46 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170315_123523_859761_C5C9D48E X-CRM114-Status: GOOD ( 16.67 ) X-BeenThere: linux-amlogic@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-amlogic@lists.infradead.org, "linux-mmc@vger.kernel.org" Sender: "linux-amlogic" Errors-To: linux-amlogic-bounces+patchwork-linux-amlogic=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Switch the driver from using a linearized bounce buffer to descriptor chain mode. This drastically improves performance. Signed-off-by: Heiner Kallweit --- drivers/mmc/host/meson-gx-mmc.c | 207 ++++++++++++++++++++++------------------ 1 file changed, 116 insertions(+), 91 deletions(-) diff --git a/drivers/mmc/host/meson-gx-mmc.c b/drivers/mmc/host/meson-gx-mmc.c index 425060da..d561065b 100644 --- a/drivers/mmc/host/meson-gx-mmc.c +++ b/drivers/mmc/host/meson-gx-mmc.c @@ -121,6 +121,13 @@ #define SD_EMMC_CFG_CMD_GAP 16 /* in clock cycles */ #define MUX_CLK_NUM_PARENTS 2 +struct sd_emmc_desc { + u32 cmd_cfg; + u32 cmd_arg; + u32 cmd_data; + u32 cmd_resp; +}; + struct meson_host { struct device *dev; struct mmc_host *mmc; @@ -136,19 +143,12 @@ struct meson_host { struct clk_divider cfg_div; struct clk *cfg_div_clk; - unsigned int bounce_buf_size; - void *bounce_buf; - dma_addr_t bounce_dma_addr; + struct sd_emmc_desc *descs; + dma_addr_t descs_dma_addr; bool vqmmc_enabled; }; -struct sd_emmc_desc { - u32 cmd_cfg; - u32 cmd_arg; - u32 cmd_data; - u32 cmd_resp; -}; #define CMD_CFG_LENGTH_SHIFT 0 #define CMD_CFG_LENGTH_MASK 0x1ff #define CMD_CFG_BLOCK_MODE BIT(9) @@ -185,6 +185,36 @@ static struct mmc_command *meson_mmc_get_next_command(struct mmc_command *cmd) return NULL; } +static enum dma_data_direction meson_mmc_get_data_dir(struct mmc_data *data) +{ + return data->flags & MMC_DATA_WRITE ? DMA_TO_DEVICE : DMA_FROM_DEVICE; +} + +static void meson_mmc_pre_req(struct mmc_host *mmc, struct mmc_request *mrq) +{ + struct mmc_data *data = mrq->data; + + if (!data) + return; + + data->host_cookie = true; + + data->sg_count = dma_map_sg(mmc_dev(mmc), data->sg, data->sg_len, + meson_mmc_get_data_dir(data)); + if (!data->sg_count) + dev_err(mmc_dev(mmc), "dma_map_sg failed"); +} + +static void meson_mmc_post_req(struct mmc_host *mmc, struct mmc_request *mrq, + int err) +{ + struct mmc_data *data = mrq->data; + + if (data && data->sg_count) + dma_unmap_sg(mmc_dev(mmc), data->sg, data->sg_len, + meson_mmc_get_data_dir(data)); +} + static int meson_mmc_clk_set(struct meson_host *host, unsigned long clk_rate) { struct mmc_host *mmc = host->mmc; @@ -434,94 +464,91 @@ static void meson_mmc_request_done(struct mmc_host *mmc, static void meson_mmc_start_cmd(struct mmc_host *mmc, struct mmc_command *cmd) { struct meson_host *host = mmc_priv(mmc); - struct sd_emmc_desc *desc, desc_tmp; - u32 cfg; - u8 blk_len, cmd_cfg_timeout; - unsigned int xfer_bytes = 0; - - /* Setup descriptors */ - dma_rmb(); - desc = &desc_tmp; - memset(desc, 0, sizeof(struct sd_emmc_desc)); + struct sd_emmc_desc *desc = host->descs; + struct mmc_data *data; + struct scatterlist *sg; + u32 cfg, cmd_cfg = 0; + u8 blk_len; + int i; - desc->cmd_cfg |= (cmd->opcode & CMD_CFG_CMD_INDEX_MASK) << - CMD_CFG_CMD_INDEX_SHIFT; - desc->cmd_cfg |= CMD_CFG_OWNER; /* owned by CPU */ - desc->cmd_arg = cmd->arg; + cmd_cfg |= (cmd->opcode & CMD_CFG_CMD_INDEX_MASK) << + CMD_CFG_CMD_INDEX_SHIFT; + cmd_cfg |= CMD_CFG_OWNER; /* owned by CPU */ /* Response */ if (cmd->flags & MMC_RSP_PRESENT) { - desc->cmd_cfg &= ~CMD_CFG_NO_RESP; + cmd_cfg &= ~CMD_CFG_NO_RESP; if (cmd->flags & MMC_RSP_136) - desc->cmd_cfg |= CMD_CFG_RESP_128; - desc->cmd_cfg |= CMD_CFG_RESP_NUM; - desc->cmd_resp = 0; + cmd_cfg |= CMD_CFG_RESP_128; + cmd_cfg |= CMD_CFG_RESP_NUM; if (!(cmd->flags & MMC_RSP_CRC)) - desc->cmd_cfg |= CMD_CFG_RESP_NOCRC; + cmd_cfg |= CMD_CFG_RESP_NOCRC; if (cmd->flags & MMC_RSP_BUSY) - desc->cmd_cfg |= CMD_CFG_R1B; + cmd_cfg |= CMD_CFG_R1B; } else { - desc->cmd_cfg |= CMD_CFG_NO_RESP; + cmd_cfg |= CMD_CFG_NO_RESP; } - /* data? */ - if (cmd->data) { - desc->cmd_cfg |= CMD_CFG_DATA_IO | CMD_CFG_BLOCK_MODE; - desc->cmd_cfg |= (cmd->data->blocks & CMD_CFG_LENGTH_MASK) << - CMD_CFG_LENGTH_SHIFT; + data = cmd->data; + if (data) { + data->bytes_xfered = 0; + cmd_cfg |= CMD_CFG_DATA_IO | CMD_CFG_BLOCK_MODE; + cmd_cfg |= ilog2(SD_EMMC_CMD_TIMEOUT_DATA) << + CMD_CFG_TIMEOUT_SHIFT; + + if (data->flags & MMC_DATA_WRITE) + cmd_cfg |= CMD_CFG_DATA_WR; /* check if block-size matches, if not update */ cfg = readl(host->regs + SD_EMMC_CFG); blk_len = cfg & (CFG_BLK_LEN_MASK << CFG_BLK_LEN_SHIFT); blk_len >>= CFG_BLK_LEN_SHIFT; - if (blk_len != ilog2(cmd->data->blksz)) { + if (blk_len != ilog2(data->blksz)) { dev_dbg(host->dev, "%s: update blk_len %d -> %d\n", - __func__, blk_len, ilog2(cmd->data->blksz)); - blk_len = ilog2(cmd->data->blksz); + __func__, blk_len, ilog2(data->blksz)); + blk_len = ilog2(data->blksz); cfg &= ~(CFG_BLK_LEN_MASK << CFG_BLK_LEN_SHIFT); cfg |= blk_len << CFG_BLK_LEN_SHIFT; writel(cfg, host->regs + SD_EMMC_CFG); } - cmd->data->bytes_xfered = 0; - xfer_bytes = cmd->data->blksz * cmd->data->blocks; - if (cmd->data->flags & MMC_DATA_WRITE) { - desc->cmd_cfg |= CMD_CFG_DATA_WR; - WARN_ON(xfer_bytes > host->bounce_buf_size); - sg_copy_to_buffer(cmd->data->sg, cmd->data->sg_len, - host->bounce_buf, xfer_bytes); - cmd->data->bytes_xfered = xfer_bytes; - dma_wmb(); - } else { - desc->cmd_cfg &= ~CMD_CFG_DATA_WR; + for_each_sg(data->sg, sg, data->sg_len, i) { + desc[i].cmd_cfg = cmd_cfg; + sg_dma_len(sg) = sg->length; + desc[i].cmd_cfg |= (sg_dma_len(sg) / data->blksz) + << CMD_CFG_LENGTH_SHIFT; + if (i > 0) + desc[i].cmd_cfg |= CMD_CFG_NO_CMD; + desc[i].cmd_arg = cmd->arg; + desc[i].cmd_resp = 0; + desc[i].cmd_data = sg_dma_address(sg); } - - desc->cmd_data = host->bounce_dma_addr & CMD_DATA_MASK; - - cmd_cfg_timeout = ilog2(SD_EMMC_CMD_TIMEOUT_DATA); + desc[data->sg_len - 1].cmd_cfg |= CMD_CFG_END_OF_CHAIN; } else { - desc->cmd_cfg &= ~CMD_CFG_DATA_IO; - cmd_cfg_timeout = ilog2(SD_EMMC_CMD_TIMEOUT); + cmd_cfg |= ilog2(SD_EMMC_CMD_TIMEOUT) << CMD_CFG_TIMEOUT_SHIFT; + cmd_cfg |= CMD_CFG_END_OF_CHAIN; + desc[0].cmd_cfg = cmd_cfg; + desc[0].cmd_arg = cmd->arg; + desc[0].cmd_resp = 0; + desc[0].cmd_data = 0; } - desc->cmd_cfg |= (cmd_cfg_timeout & CMD_CFG_TIMEOUT_MASK) << - CMD_CFG_TIMEOUT_SHIFT; host->cmd = cmd; - /* Last descriptor */ - desc->cmd_cfg |= CMD_CFG_END_OF_CHAIN; - writel(desc->cmd_cfg, host->regs + SD_EMMC_CMD_CFG); - writel(desc->cmd_data, host->regs + SD_EMMC_CMD_DAT); - writel(desc->cmd_resp, host->regs + SD_EMMC_CMD_RSP); wmb(); /* ensure descriptor is written before kicked */ - writel(desc->cmd_arg, host->regs + SD_EMMC_CMD_ARG); + cfg = host->descs_dma_addr | START_DESC_BUSY; + writel(cfg, host->regs + SD_EMMC_START); } static void meson_mmc_request(struct mmc_host *mmc, struct mmc_request *mrq) { struct meson_host *host = mmc_priv(mmc); + bool needs_pre_post_req = mrq->data && !mrq->data->host_cookie; + + if (needs_pre_post_req) + meson_mmc_pre_req(mmc, mrq); /* Stop execution */ writel(0, host->regs + SD_EMMC_START); @@ -530,6 +557,9 @@ static void meson_mmc_request(struct mmc_host *mmc, struct mmc_request *mrq) meson_mmc_start_cmd(mmc, mrq->sbc); else meson_mmc_start_cmd(mmc, mrq->cmd); + + if (needs_pre_post_req) + meson_mmc_post_req(mmc, mrq, 0); } static void meson_mmc_read_resp(struct mmc_host *mmc, struct mmc_command *cmd) @@ -550,6 +580,7 @@ static irqreturn_t meson_mmc_irq(int irq, void *dev_id) { struct meson_host *host = dev_id; struct mmc_command *cmd; + struct mmc_data *data; u32 irq_en, status, raw_status; irqreturn_t ret = IRQ_HANDLED; @@ -561,6 +592,8 @@ static irqreturn_t meson_mmc_irq(int irq, void *dev_id) if (WARN_ON(!cmd)) return IRQ_NONE; + data = cmd->data; + spin_lock(&host->lock); irq_en = readl(host->regs + SD_EMMC_IRQ_EN); raw_status = readl(host->regs + SD_EMMC_STATUS); @@ -598,12 +631,17 @@ static irqreturn_t meson_mmc_irq(int irq, void *dev_id) dev_dbg(host->dev, "Unhandled IRQ: Descriptor timeout\n"); cmd->error = -ETIMEDOUT; } + + if (data && !cmd->error) + data->bytes_xfered = data->blksz * data->blocks; + if (status & IRQ_SDIO) dev_dbg(host->dev, "Unhandled IRQ: SDIO.\n"); - if (status & (IRQ_END_OF_CHAIN | IRQ_RESP_STATUS)) - ret = IRQ_WAKE_THREAD; - else { + if (status & (IRQ_END_OF_CHAIN | IRQ_RESP_STATUS)) { + if (meson_mmc_get_next_command(cmd)) + ret = IRQ_WAKE_THREAD; + } else { dev_warn(host->dev, "Unknown IRQ! status=0x%04x: MMC CMD%u arg=0x%08x flags=0x%08x stop=%d\n", status, cmd->opcode, cmd->arg, cmd->flags, cmd->mrq->stop ? 1 : 0); @@ -632,26 +670,12 @@ static irqreturn_t meson_mmc_irq_thread(int irq, void *dev_id) { struct meson_host *host = dev_id; struct mmc_command *next_cmd, *cmd = host->cmd; - struct mmc_data *data; - unsigned int xfer_bytes; if (WARN_ON(!cmd)) return IRQ_NONE; - data = cmd->data; - if (data && data->flags & MMC_DATA_READ) { - xfer_bytes = data->blksz * data->blocks; - WARN_ON(xfer_bytes > host->bounce_buf_size); - sg_copy_from_buffer(data->sg, data->sg_len, - host->bounce_buf, xfer_bytes); - data->bytes_xfered = xfer_bytes; - } - next_cmd = meson_mmc_get_next_command(cmd); - if (next_cmd) - meson_mmc_start_cmd(host->mmc, next_cmd); - else - meson_mmc_request_done(host->mmc, cmd->mrq); + meson_mmc_start_cmd(host->mmc, next_cmd); return IRQ_HANDLED; } @@ -685,6 +709,8 @@ static const struct mmc_host_ops meson_mmc_ops = { .request = meson_mmc_request, .set_ios = meson_mmc_set_ios, .get_cd = meson_mmc_get_cd, + .pre_req = meson_mmc_pre_req, + .post_req = meson_mmc_post_req, }; static int meson_mmc_probe(struct platform_device *pdev) @@ -764,15 +790,14 @@ static int meson_mmc_probe(struct platform_device *pdev) mmc->caps |= MMC_CAP_CMD23; mmc->max_blk_count = CMD_CFG_LENGTH_MASK; - mmc->max_req_size = mmc->max_blk_count * mmc->max_blk_size; - - /* data bounce buffer */ - host->bounce_buf_size = mmc->max_req_size; - host->bounce_buf = - dma_alloc_coherent(host->dev, host->bounce_buf_size, - &host->bounce_dma_addr, GFP_KERNEL); - if (host->bounce_buf == NULL) { - dev_err(host->dev, "Unable to map allocate DMA bounce buffer.\n"); + mmc->max_segs = PAGE_SIZE / sizeof(struct sd_emmc_desc); + mmc->max_seg_size = mmc->max_blk_count * mmc->max_blk_size; + mmc->max_req_size = mmc->max_seg_size * mmc->max_segs; + + host->descs = dma_alloc_coherent(host->dev, PAGE_SIZE, + &host->descs_dma_addr, GFP_KERNEL); + if (!host->descs) { + dev_err(host->dev, "Allocating descriptor DMA buffer failed\n"); ret = -ENOMEM; goto err_div_clk; } @@ -797,8 +822,8 @@ static int meson_mmc_remove(struct platform_device *pdev) /* disable interrupts */ writel(0, host->regs + SD_EMMC_IRQ_EN); - dma_free_coherent(host->dev, host->bounce_buf_size, - host->bounce_buf, host->bounce_dma_addr); + dma_free_coherent(host->dev, PAGE_SIZE, host->descs, + host->descs_dma_addr); clk_disable_unprepare(host->cfg_div_clk); clk_disable_unprepare(host->core_clk);