From patchwork Fri Sep 6 14:31:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pascal van Leeuwen X-Patchwork-Id: 11135583 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 170761599 for ; Fri, 6 Sep 2019 15:34:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DB61020842 for ; Fri, 6 Sep 2019 15:34:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AbXjt0Qj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2394764AbfIFPem (ORCPT ); Fri, 6 Sep 2019 11:34:42 -0400 Received: from mail-ed1-f67.google.com ([209.85.208.67]:44523 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2394868AbfIFPem (ORCPT ); Fri, 6 Sep 2019 11:34:42 -0400 Received: by mail-ed1-f67.google.com with SMTP id p2so5493753edx.11 for ; Fri, 06 Sep 2019 08:34:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nuBODiQS+dIFCr86SVLtzK9jPi6yygw3exLqu21uB0c=; b=AbXjt0Qjm5oXzMCzSL/s/kF7dqOu0X8mTQ1bk3kPHzb5JCpEDvN14fuTdryNUYvdrz 4OnNfE/UMMqWUHVKjTyAlg+LxeB5rVznm3r/mb/ljPS5Ki/wJMH2iJ4A/98QEefg6A2u xBgjuS+rtzbA7bJukU2WBA7mwDGNs7Kwcigsv+VqRNMUwAUgkBTfqZu56kZ89s+EIN+2 aZVsK9YX7OrJa38LGyINFiJAxgTPiGVAZe+ZpIi2LpPhHuYH/66xu8IW8p4/GsWAZvYm 0XcN88VojtXZIQ+hZWRuw0txWaiTpGHGOGiCQyG4p9zH6Z2clSzvgoJ2L3bhysGGkJ58 PkcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=nuBODiQS+dIFCr86SVLtzK9jPi6yygw3exLqu21uB0c=; b=ckKlbmS4FCsO4cQIz9lyp06eKl0h/ZxZjqCL8Pif7e8in+kuDSppYT81DALt7BSMbq f+cqPEx8HRufEYk9n4fOQ3MLk/1EiBvn15WDaDiy47C5DBulQtxfJFYkpNvnWCjoBk8M oTAlEZQD4l1+Q0O1VJ+n3xP8+91j5Lvh5aPEerRFISyd3l4AdEiykcReHpOugflHMiLD goIOleKQvyytVlqT+OO2Ln9saOlm+G2mXuUwF7dfZVXQGLuF5Mm7Q83++1ECUyTv5nl9 5eCbabJacB5QvZE00tmeVVbGv2dS6tJZ6JwnXAZXJF3WN5Orak1YrCuyloEklrVe+9Bg 0omg== X-Gm-Message-State: APjAAAWLrvt0ywrx+fX6HVl5Utlnzztvq/PtNoa894nQSwi7FmkuIC+D xmkMzY4+tU+Orzsx7ttC3pk3iGO2 X-Google-Smtp-Source: APXvYqxbfqvq65AjdHr3Xo2A67oWSYDrYcBWJet4WzPEim3EjAdL3RHOcj8zdYsUDvrGeyJiTQMw2g== X-Received: by 2002:a05:6402:1845:: with SMTP id v5mr5194867edy.130.1567784079588; Fri, 06 Sep 2019 08:34:39 -0700 (PDT) Received: from localhost.localdomain.com ([188.204.2.113]) by smtp.gmail.com with ESMTPSA id c22sm995218eds.30.2019.09.06.08.34.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Sep 2019 08:34:39 -0700 (PDT) From: Pascal van Leeuwen X-Google-Original-From: Pascal van Leeuwen To: linux-crypto@vger.kernel.org Cc: antoine.tenart@bootlin.com, herbert@gondor.apana.org.au, davem@davemloft.net, Pascal van Leeuwen Subject: [PATCH 1/6] crypto: inside-secure - Add EIP97/EIP197 and endianness detection Date: Fri, 6 Sep 2019 16:31:48 +0200 Message-Id: <1567780313-1579-2-git-send-email-pvanleeuwen@verimatrix.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1567780313-1579-1-git-send-email-pvanleeuwen@verimatrix.com> References: <1567780313-1579-1-git-send-email-pvanleeuwen@verimatrix.com> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org This patch adds automatic EIP97/EIP197 detection, so it does not need to rely on any static value from the device table anymore. In particular, the static value from the table won't work for PCI devboards that cannot be further identified save from this direct hardware probing. The patch also adds automatic host xs endianness detection & correction. Signed-off-by: Pascal van Leeuwen --- drivers/crypto/inside-secure/safexcel.c | 154 ++++++++++++++++++++++---------- drivers/crypto/inside-secure/safexcel.h | 26 +++++- 2 files changed, 130 insertions(+), 50 deletions(-) diff --git a/drivers/crypto/inside-secure/safexcel.c b/drivers/crypto/inside-secure/safexcel.c index 2fdb188..7a37c40 100644 --- a/drivers/crypto/inside-secure/safexcel.c +++ b/drivers/crypto/inside-secure/safexcel.c @@ -393,29 +393,21 @@ static int safexcel_hw_setup_rdesc_rings(struct safexcel_crypto_priv *priv) static int safexcel_hw_init(struct safexcel_crypto_priv *priv) { - u32 version, val; + u32 val; int i, ret, pe; dev_dbg(priv->dev, "HW init: using %d pipe(s) and %d ring(s)\n", priv->config.pes, priv->config.rings); - /* Determine endianess and configure byte swap */ - version = readl(EIP197_HIA_AIC(priv) + EIP197_HIA_VERSION); - val = readl(EIP197_HIA_AIC(priv) + EIP197_HIA_MST_CTRL); - - if ((version & 0xffff) == EIP197_HIA_VERSION_BE) - val |= EIP197_MST_CTRL_BYTE_SWAP; - else if (((version >> 16) & 0xffff) == EIP197_HIA_VERSION_LE) - val |= (EIP197_MST_CTRL_NO_BYTE_SWAP >> 24); - /* * For EIP197's only set maximum number of TX commands to 2^5 = 32 * Skip for the EIP97 as it does not have this field. */ - if (priv->version != EIP97IES_MRVL) + if (priv->flags & SAFEXCEL_HW_EIP197) { + val = readl(EIP197_HIA_AIC(priv) + EIP197_HIA_MST_CTRL); val |= EIP197_MST_CTRL_TX_MAX_CMD(5); - - writel(val, EIP197_HIA_AIC(priv) + EIP197_HIA_MST_CTRL); + writel(val, EIP197_HIA_AIC(priv) + EIP197_HIA_MST_CTRL); + } /* Configure wr/rd cache values */ writel(EIP197_MST_CTRL_RD_CACHE(RD_CACHE_4BITS) | @@ -438,7 +430,7 @@ static int safexcel_hw_init(struct safexcel_crypto_priv *priv) writel(EIP197_DxE_THR_CTRL_RESET_PE, EIP197_HIA_DFE_THR(priv) + EIP197_HIA_DFE_THR_CTRL(pe)); - if (priv->version != EIP97IES_MRVL) + if (priv->flags & SAFEXCEL_HW_EIP197) /* Reset HIA input interface arbiter (EIP197 only) */ writel(EIP197_HIA_RA_PE_CTRL_RESET, EIP197_HIA_AIC(priv) + EIP197_HIA_RA_PE_CTRL(pe)); @@ -464,7 +456,7 @@ static int safexcel_hw_init(struct safexcel_crypto_priv *priv) EIP197_PE_IN_xBUF_THRES_MAX(7), EIP197_PE(priv) + EIP197_PE_IN_TBUF_THRES(pe)); - if (priv->version != EIP97IES_MRVL) + if (priv->flags & SAFEXCEL_HW_EIP197) /* enable HIA input interface arbiter and rings */ writel(EIP197_HIA_RA_PE_CTRL_EN | GENMASK(priv->config.rings - 1, 0), @@ -490,7 +482,7 @@ static int safexcel_hw_init(struct safexcel_crypto_priv *priv) /* FIXME: instability issues can occur for EIP97 but disabling * it impacts performance. */ - if (priv->version != EIP97IES_MRVL) + if (priv->flags & SAFEXCEL_HW_EIP197) val |= EIP197_HIA_DSE_CFG_EN_SINGLE_WR; writel(val, EIP197_HIA_DSE(priv) + EIP197_HIA_DSE_CFG(pe)); @@ -577,8 +569,9 @@ static int safexcel_hw_init(struct safexcel_crypto_priv *priv) /* Clear any HIA interrupt */ writel(GENMASK(30, 20), EIP197_HIA_AIC_G(priv) + EIP197_HIA_AIC_G_ACK); - if (priv->version != EIP97IES_MRVL) { + if (priv->flags & SAFEXCEL_HW_EIP197) { eip197_trc_cache_init(priv); + priv->flags |= EIP197_TRC_CACHE; ret = eip197_load_firmwares(priv); if (ret) @@ -1087,12 +1080,12 @@ static void safexcel_configure(struct safexcel_crypto_priv *priv) val = readl(EIP197_HIA_AIC_G(priv) + EIP197_HIA_OPTIONS); /* Read number of PEs from the engine */ - if (priv->version == EIP97IES_MRVL) - /* Narrow field width for EIP97 type engine */ - mask = EIP97_N_PES_MASK; - else + if (priv->flags & SAFEXCEL_HW_EIP197) /* Wider field width for all EIP197 type engines */ mask = EIP197_N_PES_MASK; + else + /* Narrow field width for EIP97 type engine */ + mask = EIP97_N_PES_MASK; priv->config.pes = (val >> EIP197_N_PES_OFFSET) & mask; @@ -1112,18 +1105,7 @@ static void safexcel_init_register_offsets(struct safexcel_crypto_priv *priv) { struct safexcel_register_offsets *offsets = &priv->offsets; - if (priv->version == EIP97IES_MRVL) { - offsets->hia_aic = EIP97_HIA_AIC_BASE; - offsets->hia_aic_g = EIP97_HIA_AIC_G_BASE; - offsets->hia_aic_r = EIP97_HIA_AIC_R_BASE; - offsets->hia_aic_xdr = EIP97_HIA_AIC_xDR_BASE; - offsets->hia_dfe = EIP97_HIA_DFE_BASE; - offsets->hia_dfe_thr = EIP97_HIA_DFE_THR_BASE; - offsets->hia_dse = EIP97_HIA_DSE_BASE; - offsets->hia_dse_thr = EIP97_HIA_DSE_THR_BASE; - offsets->hia_gen_cfg = EIP97_HIA_GEN_CFG_BASE; - offsets->pe = EIP97_PE_BASE; - } else { + if (priv->flags & SAFEXCEL_HW_EIP197) { offsets->hia_aic = EIP197_HIA_AIC_BASE; offsets->hia_aic_g = EIP197_HIA_AIC_G_BASE; offsets->hia_aic_r = EIP197_HIA_AIC_R_BASE; @@ -1134,6 +1116,19 @@ static void safexcel_init_register_offsets(struct safexcel_crypto_priv *priv) offsets->hia_dse_thr = EIP197_HIA_DSE_THR_BASE; offsets->hia_gen_cfg = EIP197_HIA_GEN_CFG_BASE; offsets->pe = EIP197_PE_BASE; + offsets->global = EIP197_GLOBAL_BASE; + } else { + offsets->hia_aic = EIP97_HIA_AIC_BASE; + offsets->hia_aic_g = EIP97_HIA_AIC_G_BASE; + offsets->hia_aic_r = EIP97_HIA_AIC_R_BASE; + offsets->hia_aic_xdr = EIP97_HIA_AIC_xDR_BASE; + offsets->hia_dfe = EIP97_HIA_DFE_BASE; + offsets->hia_dfe_thr = EIP97_HIA_DFE_THR_BASE; + offsets->hia_dse = EIP97_HIA_DSE_BASE; + offsets->hia_dse_thr = EIP97_HIA_DSE_THR_BASE; + offsets->hia_gen_cfg = EIP97_HIA_GEN_CFG_BASE; + offsets->pe = EIP97_PE_BASE; + offsets->global = EIP97_GLOBAL_BASE; } } @@ -1149,8 +1144,8 @@ static int safexcel_probe_generic(void *pdev, int is_pci_dev) { struct device *dev = priv->dev; - u32 peid; - int i, ret; + u32 peid, version, mask, val; + int i, ret, hwctg; priv->context_pool = dmam_pool_create("safexcel-context", dev, sizeof(struct safexcel_context_record), @@ -1158,23 +1153,89 @@ static int safexcel_probe_generic(void *pdev, if (!priv->context_pool) return -ENOMEM; + /* + * First try the EIP97 HIA version regs + * For the EIP197, this is guaranteed to NOT return any of the test + * values + */ + version = readl(priv->base + EIP97_HIA_AIC_BASE + EIP197_HIA_VERSION); + + mask = 0; /* do not swap */ + if (EIP197_REG_LO16(version) == EIP197_HIA_VERSION_LE) { + priv->hwconfig.hiaver = EIP197_VERSION_MASK(version); + } else if (EIP197_REG_HI16(version) == EIP197_HIA_VERSION_BE) { + /* read back byte-swapped, so complement byte swap bits */ + mask = EIP197_MST_CTRL_BYTE_SWAP_BITS; + priv->hwconfig.hiaver = EIP197_VERSION_SWAP(version); + } else { + /* So it wasn't an EIP97 ... maybe it's an EIP197? */ + version = readl(priv->base + EIP197_HIA_AIC_BASE + + EIP197_HIA_VERSION); + if (EIP197_REG_LO16(version) == EIP197_HIA_VERSION_LE) { + priv->hwconfig.hiaver = EIP197_VERSION_MASK(version); + priv->flags |= SAFEXCEL_HW_EIP197; + } else if (EIP197_REG_HI16(version) == + EIP197_HIA_VERSION_BE) { + /* read back byte-swapped, so complement swap bits */ + mask = EIP197_MST_CTRL_BYTE_SWAP_BITS; + priv->hwconfig.hiaver = EIP197_VERSION_SWAP(version); + priv->flags |= SAFEXCEL_HW_EIP197; + } else { + return -ENODEV; + } + } + + /* Now initialize the reg offsets based on the probing info so far */ safexcel_init_register_offsets(priv); + /* + * If the version was read byte-swapped, we need to flip the device + * swapping Keep in mind here, though, that what we write will also be + * byte-swapped ... + */ + if (mask) { + val = readl(EIP197_HIA_AIC(priv) + EIP197_HIA_MST_CTRL); + val = val ^ (mask >> 24); /* toggle byte swap bits */ + writel(val, EIP197_HIA_AIC(priv) + EIP197_HIA_MST_CTRL); + } + + /* + * We're not done probing yet! We may fall through to here if no HIA + * was found at all. So, with the endianness presumably correct now and + * the offsets setup, *really* probe for the EIP97/EIP197. + */ + version = readl(EIP197_GLOBAL(priv) + EIP197_VERSION); + if (((priv->flags & SAFEXCEL_HW_EIP197) && + (EIP197_REG_LO16(version) != EIP197_VERSION_LE)) || + ((!(priv->flags & SAFEXCEL_HW_EIP197) && + (EIP197_REG_LO16(version) != EIP97_VERSION_LE)))) { + /* + * We did not find the device that matched our initial probing + * (or our initial probing failed) Report appropriate error. + */ + return -ENODEV; + } + + priv->hwconfig.hwver = EIP197_VERSION_MASK(version); + hwctg = version >> 28; + peid = version & 255; + + /* Detect EIP96 packet engine and version */ + version = readl(EIP197_PE(priv) + EIP197_PE_EIP96_VERSION(0)); + if (EIP197_REG_LO16(version) != EIP96_VERSION_LE) { + dev_err(dev, "EIP%d: EIP96 not detected.\n", peid); + return -ENODEV; + } + priv->hwconfig.pever = EIP197_VERSION_MASK(version); + /* Get supported algorithms from EIP96 transform engine */ priv->hwconfig.algo_flags = readl(EIP197_PE(priv) + EIP197_PE_EIP96_OPTIONS(0)); - if (priv->version == EIP97IES_MRVL) { - peid = 97; - } else { - priv->flags |= EIP197_TRC_CACHE; - peid = 197; - } - - /* Dump some debug information important during development */ - dev_dbg(priv->dev, "Inside Secure EIP%d packetengine\n", peid); - dev_dbg(priv->dev, "Supported algorithms: %08x\n", - priv->hwconfig.algo_flags); + /* Print single info line describing what we just detected */ + dev_info(priv->dev, "EIP%d:%x(%d)-HIA:%x,PE:%x,alg:%08x\n", peid, + priv->hwconfig.hwver, hwctg, priv->hwconfig.hiaver, + priv->hwconfig.pever, priv->hwconfig.algo_flags); safexcel_configure(priv); @@ -1526,7 +1587,6 @@ void safexcel_pci_remove(struct pci_dev *pdev) { PCI_DEVICE_SUB(PCI_VENDOR_ID_XILINX, 0x9038, 0x16ae, 0xc522), - /* assume EIP197B for now */ .driver_data = EIP197_DEVBRD, }, {}, diff --git a/drivers/crypto/inside-secure/safexcel.h b/drivers/crypto/inside-secure/safexcel.h index d76a4fa..e4da706 100644 --- a/drivers/crypto/inside-secure/safexcel.h +++ b/drivers/crypto/inside-secure/safexcel.h @@ -14,8 +14,17 @@ #include #include -#define EIP197_HIA_VERSION_LE 0xca35 -#define EIP197_HIA_VERSION_BE 0x35ca +#define EIP197_HIA_VERSION_BE 0xca35 +#define EIP197_HIA_VERSION_LE 0x35ca +#define EIP97_VERSION_LE 0x9e61 +#define EIP197_VERSION_LE 0x3ac5 +#define EIP96_VERSION_LE 0x9f60 +#define EIP197_REG_LO16(reg) (reg & 0xffff) +#define EIP197_REG_HI16(reg) ((reg >> 16) & 0xffff) +#define EIP197_VERSION_MASK(reg) ((reg >> 16) & 0xfff) +#define EIP197_VERSION_SWAP(reg) (((reg & 0xf0) << 4) | \ + ((reg >> 4) & 0xf0) | \ + ((reg >> 12) & 0xf)) /* Static configuration */ #define EIP197_DEFAULT_RING_SIZE 400 @@ -70,6 +79,7 @@ #define EIP197_HIA_DSE_THR(priv) ((priv)->base + (priv)->offsets.hia_dse_thr) #define EIP197_HIA_GEN_CFG(priv) ((priv)->base + (priv)->offsets.hia_gen_cfg) #define EIP197_PE(priv) ((priv)->base + (priv)->offsets.pe) +#define EIP197_GLOBAL(priv) ((priv)->base + (priv)->offsets.global) /* EIP197 base offsets */ #define EIP197_HIA_AIC_BASE 0x90000 @@ -82,6 +92,7 @@ #define EIP197_HIA_DSE_THR_BASE 0x8d040 #define EIP197_HIA_GEN_CFG_BASE 0xf0000 #define EIP197_PE_BASE 0xa0000 +#define EIP197_GLOBAL_BASE 0xf0000 /* EIP97 base offsets */ #define EIP97_HIA_AIC_BASE 0x0 @@ -94,6 +105,7 @@ #define EIP97_HIA_DSE_THR_BASE 0xf600 #define EIP97_HIA_GEN_CFG_BASE 0x10000 #define EIP97_PE_BASE 0x10000 +#define EIP97_GLOBAL_BASE 0x10000 /* CDR/RDR register offsets */ #define EIP197_HIA_xDR_OFF(priv, r) (EIP197_HIA_AIC_xDR(priv) + (r) * 0x1000) @@ -146,9 +158,11 @@ #define EIP197_PE_EIP96_CONTEXT_CTRL(n) (0x1008 + (0x2000 * (n))) #define EIP197_PE_EIP96_CONTEXT_STAT(n) (0x100c + (0x2000 * (n))) #define EIP197_PE_EIP96_OPTIONS(n) (0x13f8 + (0x2000 * (n))) +#define EIP197_PE_EIP96_VERSION(n) (0x13fc + (0x2000 * (n))) #define EIP197_PE_OUT_DBUF_THRES(n) (0x1c00 + (0x2000 * (n))) #define EIP197_PE_OUT_TBUF_THRES(n) (0x1d00 + (0x2000 * (n))) #define EIP197_MST_CTRL 0xfff4 +#define EIP197_VERSION 0xfffc /* EIP197-specific registers, no indirection */ #define EIP197_CLASSIFICATION_RAMS 0xe0000 @@ -252,6 +266,7 @@ #define EIP197_MST_CTRL_TX_MAX_CMD(n) (((n) & 0xf) << 20) #define EIP197_MST_CTRL_BYTE_SWAP BIT(24) #define EIP197_MST_CTRL_NO_BYTE_SWAP BIT(25) +#define EIP197_MST_CTRL_BYTE_SWAP_BITS GENMASK(25, 24) /* EIP197_PE_IN_DBUF/TBUF_THRES */ #define EIP197_PE_IN_xBUF_THRES_MIN(n) ((n) << 8) @@ -652,14 +667,19 @@ struct safexcel_register_offsets { u32 hia_dse_thr; u32 hia_gen_cfg; u32 pe; + u32 global; }; enum safexcel_flags { - EIP197_TRC_CACHE = BIT(0), + EIP197_TRC_CACHE = BIT(0), + SAFEXCEL_HW_EIP197 = BIT(1), }; struct safexcel_hwconfig { enum safexcel_eip_algorithms algo_flags; + int hwver; + int hiaver; + int pever; }; struct safexcel_crypto_priv { From patchwork Fri Sep 6 14:31:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pascal van Leeuwen X-Patchwork-Id: 11135585 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7E14D1731 for ; Fri, 6 Sep 2019 15:34:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 604BE20842 for ; Fri, 6 Sep 2019 15:34:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hj8TVwX/" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404318AbfIFPem (ORCPT ); Fri, 6 Sep 2019 11:34:42 -0400 Received: from mail-ed1-f66.google.com ([209.85.208.66]:40269 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392617AbfIFPem (ORCPT ); Fri, 6 Sep 2019 11:34:42 -0400 Received: by mail-ed1-f66.google.com with SMTP id v38so6675452edm.7 for ; Fri, 06 Sep 2019 08:34:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BgOB06D1dj70xp8eWK+DKGL4PolIIHBFUxUYdIq7L9g=; b=hj8TVwX/LR6tbtdcW9chdJIkwaxJPoKP5NfjJaS8vwkz76ieefGAgCIgEMtm4xM4Po PYnArFUZmxABom7AEfeiwr0BQxGPRAcAU/vtpYjLaxDrk4xan3Uae2gvWsqaxQvJLWZD oTjIdOnU6ey+rTEy2fqHQLyBWxR/3zgIsyX9j/+VTN/MeZJrVH2tb4lU8Nf4QcG4O5oA Z6N1zBtVWz9ZlkNujHBgpng3dLYJUgBuY46BWC0sk/VxCvkVX4nHW+0QR4LIzQDKSBk9 br8h0oUemq8gvTbQYRFPPwKm0qe/BgPBtZnEM+w4h2klaRJwZYreHHfGi4XVJf2Qaoav fOHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BgOB06D1dj70xp8eWK+DKGL4PolIIHBFUxUYdIq7L9g=; b=dMsesWZESCx9EeYP3nsRd8i6QgRbqvvZjujdyBj0clXkJovS+doriQYvmOCobFxPMZ 9AWoUFv7SzPqoXud4nY9O6YA6rmUEGgqTOhZDK6gHsDeAiCkOJsrocGMG+z7iP1UydW0 ghqNn7y7lDoFbYHuvCiTjAO/AckSr5vTnFp21b6QB/LmFlQX/apdCZZpOR7Wc3I9wq5n 0tb/F+FmcrcjvD+VSbNc/REeqC5pysOGlLvdCEB3+te9fev+XAMlAZ+0cFH5YGy7mIuQ i+9TJhah31ZJxFExlnHKByWgYRgL1M8CCGpBmnFb1LwIS17n5L92lgOBCCjRpCej3vWL GLfw== X-Gm-Message-State: APjAAAV0KI2VW/LXencsoQdBVi/Ni5hr8wQmBlXwL0NPY0aC3ahy0NVr ZlMB4cDeI186H4GI7okTaq5sjPhM X-Google-Smtp-Source: APXvYqym9I9mnEvQM/9UfyAT/InRl95B+tztZatEI0JOO3OOQY/ajUwlPzDcknxzi9RDsOrACnQeCA== X-Received: by 2002:a05:6402:a50:: with SMTP id bt16mr10493857edb.287.1567784080709; Fri, 06 Sep 2019 08:34:40 -0700 (PDT) Received: from localhost.localdomain.com ([188.204.2.113]) by smtp.gmail.com with ESMTPSA id c22sm995218eds.30.2019.09.06.08.34.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Sep 2019 08:34:39 -0700 (PDT) From: Pascal van Leeuwen X-Google-Original-From: Pascal van Leeuwen To: linux-crypto@vger.kernel.org Cc: antoine.tenart@bootlin.com, herbert@gondor.apana.org.au, davem@davemloft.net, Pascal van Leeuwen Subject: [PATCH 2/6] crypto: inside-secure: Corrected configuration of EIP96_TOKEN_CTRL Date: Fri, 6 Sep 2019 16:31:49 +0200 Message-Id: <1567780313-1579-3-git-send-email-pvanleeuwen@verimatrix.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1567780313-1579-1-git-send-email-pvanleeuwen@verimatrix.com> References: <1567780313-1579-1-git-send-email-pvanleeuwen@verimatrix.com> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org This patch corrects the configuration of the EIP197_PE_EIP96_TOKEN_CTRL register. Previous value was wrong and potentially dangerous. Signed-off-by: Pascal van Leeuwen --- drivers/crypto/inside-secure/safexcel.c | 4 ++-- drivers/crypto/inside-secure/safexcel.h | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/crypto/inside-secure/safexcel.c b/drivers/crypto/inside-secure/safexcel.c index 7a37c40..d699827 100644 --- a/drivers/crypto/inside-secure/safexcel.c +++ b/drivers/crypto/inside-secure/safexcel.c @@ -498,8 +498,8 @@ static int safexcel_hw_init(struct safexcel_crypto_priv *priv) /* Token & context configuration */ val = EIP197_PE_EIP96_TOKEN_CTRL_CTX_UPDATES | - EIP197_PE_EIP96_TOKEN_CTRL_REUSE_CTX | - EIP197_PE_EIP96_TOKEN_CTRL_POST_REUSE_CTX; + EIP197_PE_EIP96_TOKEN_CTRL_NO_TOKEN_WAIT | + EIP197_PE_EIP96_TOKEN_CTRL_ENABLE_TIMEOUT; writel(val, EIP197_PE(priv) + EIP197_PE_EIP96_TOKEN_CTRL(pe)); /* H/W capabilities selection: just enable everything */ diff --git a/drivers/crypto/inside-secure/safexcel.h b/drivers/crypto/inside-secure/safexcel.h index e4da706..10a96dc 100644 --- a/drivers/crypto/inside-secure/safexcel.h +++ b/drivers/crypto/inside-secure/safexcel.h @@ -296,8 +296,8 @@ /* EIP197_PE_EIP96_TOKEN_CTRL */ #define EIP197_PE_EIP96_TOKEN_CTRL_CTX_UPDATES BIT(16) -#define EIP197_PE_EIP96_TOKEN_CTRL_REUSE_CTX BIT(19) -#define EIP197_PE_EIP96_TOKEN_CTRL_POST_REUSE_CTX BIT(20) +#define EIP197_PE_EIP96_TOKEN_CTRL_NO_TOKEN_WAIT BIT(17) +#define EIP197_PE_EIP96_TOKEN_CTRL_ENABLE_TIMEOUT BIT(22) /* EIP197_PE_EIP96_FUNCTION_EN */ #define EIP197_FUNCTION_ALL 0xffffffff From patchwork Fri Sep 6 14:31:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pascal van Leeuwen X-Patchwork-Id: 11135587 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AF33514ED for ; Fri, 6 Sep 2019 15:34:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9003020838 for ; Fri, 6 Sep 2019 15:34:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jwJnIJk0" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404468AbfIFPeo (ORCPT ); Fri, 6 Sep 2019 11:34:44 -0400 Received: from mail-ed1-f67.google.com ([209.85.208.67]:46957 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404107AbfIFPen (ORCPT ); Fri, 6 Sep 2019 11:34:43 -0400 Received: by mail-ed1-f67.google.com with SMTP id i8so6613624edn.13 for ; Fri, 06 Sep 2019 08:34:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BQ/tbFL5AfHCrTZp1kZQ0W5BcggcoXkkhGsgCLGVSkM=; b=jwJnIJk06E/j8p+jcflucZnYHBzsUyVx+qT1xiQMVmT2Q270eI98u9GSPAZRnm/AGG +35VICH/l8LUp7dU37sj98YRR6JaBCIBvw7ZLK+4M3OmTlnHvD2vGJ5hoWaCcRc4kFBK DxWPtrDu4XTjhvn9NPjINYkkG3y1CwnT47khV5IRADgbmJplE7nVcTbJ0jFylLyCT902 +zkKUNFL2sU/uBRXV58FLNzd8jFpOZN0neUfRy5FiNilcMcYZvwn6iqRkC7O2yTh5ju1 /7t+idTgdhjX7nDoODxFCpVUBGkG9F5xoW/l6bwtWrALRH5lkRRzplvw2YXNAawjx6fr hUQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BQ/tbFL5AfHCrTZp1kZQ0W5BcggcoXkkhGsgCLGVSkM=; b=cGOczjXWTRuEZqVZTnzoBN6KzDFVb3x4tliqtJzSl8uSjT5YB5KWf8SCpPihA2NBjf ls+UMHZ8jG5XpPQ4f/k8QHytSeWOcucy/GzZtTkkllpLJNz5j0GXjt2Kuu3B7VX3Xcb9 u73g2Nkm9f6xL1WskZx1yVUiqfwfboT0tgtK1HFAlQ36uRpG+Xs2IXJeCA8axeFP17rl 2RZjDCAV2P9G87R2SurK/MMniOuu8Sk1k62AjbZ6vApcbC90gijnl4n6pNsNSt2sO2oP ppuuDlZJt1yqYOJM3kD6/9h9vSqZzS1ak9PXa1CQp+xvT0QOO+cS2eXxsBc5WRnZMSM7 aB7Q== X-Gm-Message-State: APjAAAXOYRBKuuK2Aa54Dy5ZKw/e0NMxw8CiylH1kHdOOYcIrIOkXBdd g+k40NXdBwTZjnOQnjhyVfOa3S6Y X-Google-Smtp-Source: APXvYqyKyPEM09Krs+KXjFOBf5uDd63DKvgz4W1YfOpI7MGvN5h4t6/cGtLmRX9+TrXe6/zenUNMjg== X-Received: by 2002:a05:6402:611:: with SMTP id n17mr10500238edv.33.1567784081523; Fri, 06 Sep 2019 08:34:41 -0700 (PDT) Received: from localhost.localdomain.com ([188.204.2.113]) by smtp.gmail.com with ESMTPSA id c22sm995218eds.30.2019.09.06.08.34.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Sep 2019 08:34:40 -0700 (PDT) From: Pascal van Leeuwen X-Google-Original-From: Pascal van Leeuwen To: linux-crypto@vger.kernel.org Cc: antoine.tenart@bootlin.com, herbert@gondor.apana.org.au, davem@davemloft.net, Pascal van Leeuwen Subject: [PATCH 3/6] crypto: inside-secure - Enable extended algorithms on newer HW Date: Fri, 6 Sep 2019 16:31:50 +0200 Message-Id: <1567780313-1579-4-git-send-email-pvanleeuwen@verimatrix.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1567780313-1579-1-git-send-email-pvanleeuwen@verimatrix.com> References: <1567780313-1579-1-git-send-email-pvanleeuwen@verimatrix.com> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org This patch enables algorithms that did not fit the original 32 bit FUNCTION_EN register anymore via the FUNCTION2_EN extension reg. Signed-off-by: Pascal van Leeuwen --- drivers/crypto/inside-secure/safexcel.c | 2 ++ drivers/crypto/inside-secure/safexcel.h | 1 + 2 files changed, 3 insertions(+) diff --git a/drivers/crypto/inside-secure/safexcel.c b/drivers/crypto/inside-secure/safexcel.c index d699827..d9b927b 100644 --- a/drivers/crypto/inside-secure/safexcel.c +++ b/drivers/crypto/inside-secure/safexcel.c @@ -505,6 +505,8 @@ static int safexcel_hw_init(struct safexcel_crypto_priv *priv) /* H/W capabilities selection: just enable everything */ writel(EIP197_FUNCTION_ALL, EIP197_PE(priv) + EIP197_PE_EIP96_FUNCTION_EN(pe)); + writel(EIP197_FUNCTION_ALL, + EIP197_PE(priv) + EIP197_PE_EIP96_FUNCTION2_EN(pe)); } /* Command Descriptor Rings prepare */ diff --git a/drivers/crypto/inside-secure/safexcel.h b/drivers/crypto/inside-secure/safexcel.h index 10a96dc..e9bda97 100644 --- a/drivers/crypto/inside-secure/safexcel.h +++ b/drivers/crypto/inside-secure/safexcel.h @@ -157,6 +157,7 @@ #define EIP197_PE_EIP96_FUNCTION_EN(n) (0x1004 + (0x2000 * (n))) #define EIP197_PE_EIP96_CONTEXT_CTRL(n) (0x1008 + (0x2000 * (n))) #define EIP197_PE_EIP96_CONTEXT_STAT(n) (0x100c + (0x2000 * (n))) +#define EIP197_PE_EIP96_FUNCTION2_EN(n) (0x1030 + (0x2000 * (n))) #define EIP197_PE_EIP96_OPTIONS(n) (0x13f8 + (0x2000 * (n))) #define EIP197_PE_EIP96_VERSION(n) (0x13fc + (0x2000 * (n))) #define EIP197_PE_OUT_DBUF_THRES(n) (0x1c00 + (0x2000 * (n))) From patchwork Fri Sep 6 14:31:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pascal van Leeuwen X-Patchwork-Id: 11135589 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CC95F1599 for ; Fri, 6 Sep 2019 15:34:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A4F2820842 for ; Fri, 6 Sep 2019 15:34:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iUnXUXuP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404107AbfIFPep (ORCPT ); Fri, 6 Sep 2019 11:34:45 -0400 Received: from mail-ed1-f66.google.com ([209.85.208.66]:32859 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404387AbfIFPeo (ORCPT ); Fri, 6 Sep 2019 11:34:44 -0400 Received: by mail-ed1-f66.google.com with SMTP id o9so6722803edq.0 for ; Fri, 06 Sep 2019 08:34:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=6uMmV5ErfzayV8PcAf3sreZod7mbrSgyuBteR9K0XjM=; b=iUnXUXuPQ5mP54G3HVvx2zp22zX+CvOqlrh2Ct+vE+0Cb4FsXLIhw2tXb+zW6oGhGW UNb8DBXKEeg5nCgCjTDQTmrlMeXW4NvEsAlt5JoaYSLPMOFJjG7KadOOcKfMm5FYAI0R rYF67HpbRPS0Xd18H09XrC8AgnUpLC1ztCX4qr6RbevLRUs3I7rn2VeDu0iPpbNa6q7G QKLkI0N6/SFbMiZT+mVLfQskYOe0kITockysvGv7jFMm3HDKhEPQHMUkFmzJUlzAOHcT WqDhz9VawySVdXgvbN0Jfnf+tJurOZFpoRBMlsbYfFHCBGjcb7nWQlpjls2WbHw4L+DG 4uPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=6uMmV5ErfzayV8PcAf3sreZod7mbrSgyuBteR9K0XjM=; b=on/hignGaQBrNvPl4+otRCJ2YcgeZczd5gWp+GilEsfoWtzsxjdQ4SRKqM2OTwJqek V9ylV8slYNJy38dIYYOFYtdHbtzfgqB95M98F1wh9cREQcC7PC+dhFFWvsA6nN6zegzU VdKbOZuPTFgZilnU1f5GeBcsl4gVF8LrZESVdC+9uidEspJXxNML1SYZQ4N1+of9ZCFv H8yR2WrSdgz4QCC+z5hLU/RxAOD//46sjGguhvNiItNGIK2pp4UKma4Ix5SrEo7kCSUy qiv5zt9zkRt/RXRp1j/2CfMXo1FQDYoxn5JneOh8jMgNa16Kdq457Yp8T1uRUd11IHpQ 0+ew== X-Gm-Message-State: APjAAAVk29cEScc0O7NoEJMlHI+6kibvFq9+DzO1m9wKbmwOd34Vf56q Ep8e9BrRLf3/7KCM+Xr9ngtjOQOD X-Google-Smtp-Source: APXvYqzrxTTHEmeYWXELzinfReP9UtPwQ3xxzZbC2tRADVl5ssCBGkq8ICM6/DnbP7v1jV43FG3djg== X-Received: by 2002:a50:9e26:: with SMTP id z35mr10219379ede.265.1567784082228; Fri, 06 Sep 2019 08:34:42 -0700 (PDT) Received: from localhost.localdomain.com ([188.204.2.113]) by smtp.gmail.com with ESMTPSA id c22sm995218eds.30.2019.09.06.08.34.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Sep 2019 08:34:41 -0700 (PDT) From: Pascal van Leeuwen X-Google-Original-From: Pascal van Leeuwen To: linux-crypto@vger.kernel.org Cc: antoine.tenart@bootlin.com, herbert@gondor.apana.org.au, davem@davemloft.net, Pascal van Leeuwen Subject: [PATCH 4/6] crypto: inside-secure - Base CD fetchcount on actual CD FIFO size Date: Fri, 6 Sep 2019 16:31:51 +0200 Message-Id: <1567780313-1579-5-git-send-email-pvanleeuwen@verimatrix.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1567780313-1579-1-git-send-email-pvanleeuwen@verimatrix.com> References: <1567780313-1579-1-git-send-email-pvanleeuwen@verimatrix.com> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org This patch derives the command descriptor fetch count from the actual FIFO size advertised by the hardware. Fetching command descriptors one at a time is a performance bottleneck for small blocks, especially on hardware with multiple pipes. Even moreso if the HW has few rings. Signed-off-by: Pascal van Leeuwen --- drivers/crypto/inside-secure/safexcel.c | 47 ++++++++++++++++++++++++++------- drivers/crypto/inside-secure/safexcel.h | 11 ++++++++ 2 files changed, 48 insertions(+), 10 deletions(-) diff --git a/drivers/crypto/inside-secure/safexcel.c b/drivers/crypto/inside-secure/safexcel.c index d9b927b..9384491 100644 --- a/drivers/crypto/inside-secure/safexcel.c +++ b/drivers/crypto/inside-secure/safexcel.c @@ -310,13 +310,22 @@ static int eip197_load_firmwares(struct safexcel_crypto_priv *priv) static int safexcel_hw_setup_cdesc_rings(struct safexcel_crypto_priv *priv) { u32 hdw, cd_size_rnd, val; - int i; + int i, cd_fetch_cnt; - hdw = readl(EIP197_HIA_AIC_G(priv) + EIP197_HIA_OPTIONS); - hdw &= GENMASK(27, 25); - hdw >>= 25; - - cd_size_rnd = (priv->config.cd_size + (BIT(hdw) - 1)) >> hdw; + cd_size_rnd = (priv->config.cd_size + + (BIT(priv->hwconfig.hwdataw) - 1)) >> + priv->hwconfig.hwdataw; + /* determine number of CD's we can fetch into the CD FIFO as 1 block */ + if (priv->flags & SAFEXCEL_HW_EIP197) { + /* EIP197: try to fetch enough in 1 go to keep all pipes busy */ + cd_fetch_cnt = (1 << priv->hwconfig.hwcfsize) / cd_size_rnd; + cd_fetch_cnt = min_t(uint, cd_fetch_cnt, + (priv->config.pes * EIP197_FETCH_DEPTH)); + } else { + /* for the EIP97, just fetch all that fits minus 1 */ + cd_fetch_cnt = ((1 << priv->hwconfig.hwcfsize) / + cd_size_rnd) - 1; + } for (i = 0; i < priv->config.rings; i++) { /* ring base address */ @@ -328,8 +337,8 @@ static int safexcel_hw_setup_cdesc_rings(struct safexcel_crypto_priv *priv) writel(EIP197_xDR_DESC_MODE_64BIT | (priv->config.cd_offset << 16) | priv->config.cd_size, EIP197_HIA_CDR(priv, i) + EIP197_HIA_xDR_DESC_SIZE); - writel(((EIP197_FETCH_COUNT * (cd_size_rnd << hdw)) << 16) | - (EIP197_FETCH_COUNT * priv->config.cd_offset), + writel(((cd_fetch_cnt * (cd_size_rnd << hdw)) << 16) | + (cd_fetch_cnt * priv->config.cd_offset), EIP197_HIA_CDR(priv, i) + EIP197_HIA_xDR_CFG); /* Configure DMA tx control */ @@ -1146,7 +1155,7 @@ static int safexcel_probe_generic(void *pdev, int is_pci_dev) { struct device *dev = priv->dev; - u32 peid, version, mask, val; + u32 peid, version, mask, val, hiaopt; int i, ret, hwctg; priv->context_pool = dmam_pool_create("safexcel-context", dev, @@ -1230,13 +1239,31 @@ static int safexcel_probe_generic(void *pdev, } priv->hwconfig.pever = EIP197_VERSION_MASK(version); + hiaopt = readl(EIP197_HIA_AIC(priv) + EIP197_HIA_OPTIONS); + + if (priv->flags & SAFEXCEL_HW_EIP197) { + /* EIP197 */ + priv->hwconfig.hwdataw = (hiaopt >> EIP197_HWDATAW_OFFSET) & + EIP197_HWDATAW_MASK; + priv->hwconfig.hwcfsize = ((hiaopt >> EIP197_CFSIZE_OFFSET) & + EIP197_CFSIZE_MASK) + + EIP197_CFSIZE_ADJUST; + } else { + /* EIP97 */ + priv->hwconfig.hwdataw = (hiaopt >> EIP197_HWDATAW_OFFSET) & + EIP97_HWDATAW_MASK; + priv->hwconfig.hwcfsize = (hiaopt >> EIP97_CFSIZE_OFFSET) & + EIP97_CFSIZE_MASK; + } + /* Get supported algorithms from EIP96 transform engine */ priv->hwconfig.algo_flags = readl(EIP197_PE(priv) + EIP197_PE_EIP96_OPTIONS(0)); /* Print single info line describing what we just detected */ - dev_info(priv->dev, "EIP%d:%x(%d)-HIA:%x,PE:%x,alg:%08x\n", peid, + dev_info(priv->dev, "EIP%d:%x(%d)-HIA:%x(%d,%d),PE:%x,alg:%08x\n", peid, priv->hwconfig.hwver, hwctg, priv->hwconfig.hiaver, + priv->hwconfig.hwdataw, priv->hwconfig.hwcfsize, priv->hwconfig.pever, priv->hwconfig.algo_flags); safexcel_configure(priv); diff --git a/drivers/crypto/inside-secure/safexcel.h b/drivers/crypto/inside-secure/safexcel.h index e9bda97..e01aa70 100644 --- a/drivers/crypto/inside-secure/safexcel.h +++ b/drivers/crypto/inside-secure/safexcel.h @@ -31,6 +31,7 @@ #define EIP197_MAX_TOKENS 18 #define EIP197_MAX_RINGS 4 #define EIP197_FETCH_COUNT 1 +#define EIP197_FETCH_DEPTH 2 #define EIP197_MAX_BATCH_SZ 64 #define EIP197_GFP_FLAGS(base) ((base).flags & CRYPTO_TFM_REQ_MAY_SLEEP ? \ @@ -225,6 +226,14 @@ #define EIP197_N_PES_OFFSET 4 #define EIP197_N_PES_MASK GENMASK(4, 0) #define EIP97_N_PES_MASK GENMASK(2, 0) +#define EIP197_HWDATAW_OFFSET 25 +#define EIP197_HWDATAW_MASK GENMASK(3, 0) +#define EIP97_HWDATAW_MASK GENMASK(2, 0) +#define EIP197_CFSIZE_OFFSET 9 +#define EIP197_CFSIZE_ADJUST 4 +#define EIP97_CFSIZE_OFFSET 8 +#define EIP197_CFSIZE_MASK GENMASK(3, 0) +#define EIP97_CFSIZE_MASK GENMASK(4, 0) /* EIP197_HIA_AIC_R_ENABLE_CTRL */ #define EIP197_CDR_IRQ(n) BIT((n) * 2) @@ -681,6 +690,8 @@ struct safexcel_hwconfig { int hwver; int hiaver; int pever; + int hwdataw; + int hwcfsize; }; struct safexcel_crypto_priv { From patchwork Fri Sep 6 14:31:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pascal van Leeuwen X-Patchwork-Id: 11135591 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0E60A1731 for ; Fri, 6 Sep 2019 15:34:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E3A2E20842 for ; Fri, 6 Sep 2019 15:34:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lArlZ7BH" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404387AbfIFPep (ORCPT ); Fri, 6 Sep 2019 11:34:45 -0400 Received: from mail-ed1-f68.google.com ([209.85.208.68]:42785 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404459AbfIFPep (ORCPT ); Fri, 6 Sep 2019 11:34:45 -0400 Received: by mail-ed1-f68.google.com with SMTP id y91so6654227ede.9 for ; Fri, 06 Sep 2019 08:34:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=mo136QagH5i3b7/V+ae5K7cs0AQXi+Ek+lBv3k/aE9E=; b=lArlZ7BHMsd+bW+HM9w5C3YNw8/4yB5Em0ZskRUrHV07c+RMzNvRt3Y4PZA4p6i8Jj n1dn6JatcEquh7ffJK5BbRhxW3Yuee7wS49ltKZ053t+Gb34k+s+hjCD49nC+znTz3nx tJoj0BPNAHNArYFCrIgoPd3wX/jeD4n2dDPds0OMrHlPX01R5dBKnqaV2gWnkQykUeS8 zstyfvFFogDOIMwP9p0xeyXaLBz1Xx6Lk7A3OT6n2tRgXcUqzDJiXQCjOUGXyrEdHglE S5QB7klVZwu+8wUjnHpspJ+1LwARztUO+cCXJ8Ri/bwQ4omSonUObux1HnwDxhucT9YB lreQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=mo136QagH5i3b7/V+ae5K7cs0AQXi+Ek+lBv3k/aE9E=; b=jkxfd5HS0+UNAdc+G5tR25HQaJWJ8K5pScmMJuFX6eTVP8xOCGctyEwkorecrsoVKU wQ43rzMOYSiKeNnpfJ9fe3zGvtx93d5U903B23UUDqajREwrB/0n6zRDiUN9apIiniEO 1XPBSY5mapM4LCTZivOMPl8u2khtn5YR3YDYvgC0KakD4sSJo8Sawc/g1DVAkbXjQxqy 84C2WxLADD8T1OiJhVpbG+C1OySd5zOjlILLYtU3cV4y7c8INxYFvUTWHRCyADf3GAYI bOlzxuszTwdnOZf0aH6fFOA9+ugWrgMsxDLSjgTMKgOioX9PNVs1xWgiPnWT79uqJj/D HiDw== X-Gm-Message-State: APjAAAWlGB3l5PplOicMB7LQC43i8XeJ5Gw56Oo2KsWSBF+EFmIsUWQZ wpP8HZKN5sZ5hB+InPuak88IjPGX X-Google-Smtp-Source: APXvYqxRNCgw6jU8i+fzYjp2mLV05YtUprst0BnFLYPqRuaRihmBBbzhuGZ12xqgEtmatQLwIIG/mw== X-Received: by 2002:a50:95a3:: with SMTP id w32mr10321989eda.211.1567784083114; Fri, 06 Sep 2019 08:34:43 -0700 (PDT) Received: from localhost.localdomain.com ([188.204.2.113]) by smtp.gmail.com with ESMTPSA id c22sm995218eds.30.2019.09.06.08.34.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Sep 2019 08:34:42 -0700 (PDT) From: Pascal van Leeuwen X-Google-Original-From: Pascal van Leeuwen To: linux-crypto@vger.kernel.org Cc: antoine.tenart@bootlin.com, herbert@gondor.apana.org.au, davem@davemloft.net, Pascal van Leeuwen Subject: [PATCH 5/6] crypto: inside-secure - Base RD fetchcount on actual RD FIFO size Date: Fri, 6 Sep 2019 16:31:52 +0200 Message-Id: <1567780313-1579-6-git-send-email-pvanleeuwen@verimatrix.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1567780313-1579-1-git-send-email-pvanleeuwen@verimatrix.com> References: <1567780313-1579-1-git-send-email-pvanleeuwen@verimatrix.com> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org This patch derives the result descriptor fetch count from the actual FIFO size advertised by the hardware. Fetching result descriptors one at a time is a performance bottleneck for small blocks, especially on hardware with multiple pipes. Even moreso if the HW has few rings. Signed-off-by: Pascal van Leeuwen --- drivers/crypto/inside-secure/safexcel.c | 37 +++++++++++++++++++++++---------- drivers/crypto/inside-secure/safexcel.h | 15 ++++++++++++- 2 files changed, 40 insertions(+), 12 deletions(-) diff --git a/drivers/crypto/inside-secure/safexcel.c b/drivers/crypto/inside-secure/safexcel.c index 9384491..a607786 100644 --- a/drivers/crypto/inside-secure/safexcel.c +++ b/drivers/crypto/inside-secure/safexcel.c @@ -357,13 +357,22 @@ static int safexcel_hw_setup_cdesc_rings(struct safexcel_crypto_priv *priv) static int safexcel_hw_setup_rdesc_rings(struct safexcel_crypto_priv *priv) { u32 hdw, rd_size_rnd, val; - int i; - - hdw = readl(EIP197_HIA_AIC_G(priv) + EIP197_HIA_OPTIONS); - hdw &= GENMASK(27, 25); - hdw >>= 25; + int i, rd_fetch_cnt; - rd_size_rnd = (priv->config.rd_size + (BIT(hdw) - 1)) >> hdw; + /* determine number of RD's we can fetch into the FIFO as one block */ + rd_size_rnd = (EIP197_RD64_FETCH_SIZE + + BIT(priv->hwconfig.hwdataw) - 1) >> + priv->hwconfig.hwdataw; + if (priv->flags & SAFEXCEL_HW_EIP197) { + /* EIP197: try to fetch enough in 1 go to keep all pipes busy */ + rd_fetch_cnt = (1 << priv->hwconfig.hwrfsize) / rd_size_rnd; + rd_fetch_cnt = min_t(uint, rd_fetch_cnt, + (priv->config.pes * EIP197_FETCH_DEPTH)); + } else { + /* for the EIP97, just fetch all that fits minus 1 */ + rd_fetch_cnt = ((1 << priv->hwconfig.hwrfsize) / + rd_size_rnd) - 1; + } for (i = 0; i < priv->config.rings; i++) { /* ring base address */ @@ -376,8 +385,8 @@ static int safexcel_hw_setup_rdesc_rings(struct safexcel_crypto_priv *priv) priv->config.rd_size, EIP197_HIA_RDR(priv, i) + EIP197_HIA_xDR_DESC_SIZE); - writel(((EIP197_FETCH_COUNT * (rd_size_rnd << hdw)) << 16) | - (EIP197_FETCH_COUNT * priv->config.rd_offset), + writel(((rd_fetch_cnt * (rd_size_rnd << hdw)) << 16) | + (rd_fetch_cnt * priv->config.rd_offset), EIP197_HIA_RDR(priv, i) + EIP197_HIA_xDR_CFG); /* Configure DMA tx control */ @@ -1248,12 +1257,17 @@ static int safexcel_probe_generic(void *pdev, priv->hwconfig.hwcfsize = ((hiaopt >> EIP197_CFSIZE_OFFSET) & EIP197_CFSIZE_MASK) + EIP197_CFSIZE_ADJUST; + priv->hwconfig.hwrfsize = ((hiaopt >> EIP197_RFSIZE_OFFSET) & + EIP197_RFSIZE_MASK) + + EIP197_RFSIZE_ADJUST; } else { /* EIP97 */ priv->hwconfig.hwdataw = (hiaopt >> EIP197_HWDATAW_OFFSET) & EIP97_HWDATAW_MASK; priv->hwconfig.hwcfsize = (hiaopt >> EIP97_CFSIZE_OFFSET) & EIP97_CFSIZE_MASK; + priv->hwconfig.hwrfsize = (hiaopt >> EIP97_RFSIZE_OFFSET) & + EIP97_RFSIZE_MASK; } /* Get supported algorithms from EIP96 transform engine */ @@ -1261,10 +1275,11 @@ static int safexcel_probe_generic(void *pdev, EIP197_PE_EIP96_OPTIONS(0)); /* Print single info line describing what we just detected */ - dev_info(priv->dev, "EIP%d:%x(%d)-HIA:%x(%d,%d),PE:%x,alg:%08x\n", peid, - priv->hwconfig.hwver, hwctg, priv->hwconfig.hiaver, + dev_info(priv->dev, "EIP%d:%x(%d)-HIA:%x(%d,%d,%d),PE:%x,alg:%08x\n", + peid, priv->hwconfig.hwver, hwctg, priv->hwconfig.hiaver, priv->hwconfig.hwdataw, priv->hwconfig.hwcfsize, - priv->hwconfig.pever, priv->hwconfig.algo_flags); + priv->hwconfig.hwrfsize, priv->hwconfig.pever, + priv->hwconfig.algo_flags); safexcel_configure(priv); diff --git a/drivers/crypto/inside-secure/safexcel.h b/drivers/crypto/inside-secure/safexcel.h index e01aa70..19049f1 100644 --- a/drivers/crypto/inside-secure/safexcel.h +++ b/drivers/crypto/inside-secure/safexcel.h @@ -30,7 +30,6 @@ #define EIP197_DEFAULT_RING_SIZE 400 #define EIP197_MAX_TOKENS 18 #define EIP197_MAX_RINGS 4 -#define EIP197_FETCH_COUNT 1 #define EIP197_FETCH_DEPTH 2 #define EIP197_MAX_BATCH_SZ 64 @@ -234,6 +233,11 @@ #define EIP97_CFSIZE_OFFSET 8 #define EIP197_CFSIZE_MASK GENMASK(3, 0) #define EIP97_CFSIZE_MASK GENMASK(4, 0) +#define EIP197_RFSIZE_OFFSET 12 +#define EIP197_RFSIZE_ADJUST 4 +#define EIP97_RFSIZE_OFFSET 12 +#define EIP197_RFSIZE_MASK GENMASK(3, 0) +#define EIP97_RFSIZE_MASK GENMASK(4, 0) /* EIP197_HIA_AIC_R_ENABLE_CTRL */ #define EIP197_CDR_IRQ(n) BIT((n) * 2) @@ -463,6 +467,14 @@ struct safexcel_result_desc { struct result_data_desc result_data; } __packed; +/* + * The EIP(1)97 only needs to fetch the descriptor part of + * the result descriptor, not the result token part! + */ +#define EIP197_RD64_FETCH_SIZE ((sizeof(struct safexcel_result_desc) -\ + sizeof(struct result_data_desc)) /\ + sizeof(u32)) + struct safexcel_token { u32 packet_length:17; u8 stat:2; @@ -692,6 +704,7 @@ struct safexcel_hwconfig { int pever; int hwdataw; int hwcfsize; + int hwrfsize; }; struct safexcel_crypto_priv { From patchwork Fri Sep 6 14:31:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pascal van Leeuwen X-Patchwork-Id: 11135593 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 84DE914ED for ; Fri, 6 Sep 2019 15:34:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 545F920842 for ; Fri, 6 Sep 2019 15:34:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Q23wTPyo" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404459AbfIFPer (ORCPT ); Fri, 6 Sep 2019 11:34:47 -0400 Received: from mail-ed1-f68.google.com ([209.85.208.68]:42787 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404530AbfIFPer (ORCPT ); Fri, 6 Sep 2019 11:34:47 -0400 Received: by mail-ed1-f68.google.com with SMTP id y91so6654268ede.9 for ; Fri, 06 Sep 2019 08:34:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dAAzGIhCxm/LOlY3aIkQquAnjHh7ebzmiAzVPFgI/KM=; b=Q23wTPyo/IDB0k86RcU1LdA6PC6GQVp/ymINwkN9vWPzrchWqGqaHxq2SHOvgU0cnk 2aebNFHszd8If63VmgQSUEVhu288E3K4t5Yl+i5gSJ/2EYDQ3cHU1Fu915r9Le37Lgc8 EgLfC6jX9QIXiAHpth4VOeKsXNHVrJ8VDeagci5GVPxpU9jPunsH0lJQeRMRkdupgDyT zgGP/CBxMmAPmGK5AlYOxzGnI7IKETuiTXlLuYk95nRUbIECT9jFxw698eomXDRvjsGj pqGULd89fR4OjWYxGUw6worQqCXTXJsJVI2TCaPMVsbLm3J1k9SHxL5+QhYRdvCMM1PM j9lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dAAzGIhCxm/LOlY3aIkQquAnjHh7ebzmiAzVPFgI/KM=; b=VjA1CelaJAh/BnL4NwRFRl21GVULh11Sq6A/nX7jrNFygWMb3blsOVzAkRuyB4uuHd e7/3aTnlXU1E70pXeL3u2MClA4TN716IvOtI2FcXqON2r2zDv0BUEJQAbIFqSECBo68K cibtHU0Vajt7aeZQ9kO1EdO3GTZMUOVT+BaQcVNOLpNlIav2gEMVPkLEe5sHgCDLdHFL obQ4kJ4k6aGfSv0ucxihG27Yh1pRBDe4WHFiVCVKulvBPlX+o/3DmkkWjpwzinxDcBto fuUVb/+htPaJ2CsekKIeMhcw193uXOXlfg/gnV3ViAq3gaHS+qCETAPON9bBlGovNU0+ vsdA== X-Gm-Message-State: APjAAAVS626kXwHwY/W4imrVABKaLQj+D9dWwng+M05jh2zNR1RlyCr0 5i3kchgm3adsOZVhv6K/A4oi/9As X-Google-Smtp-Source: APXvYqwJ2Sj3LL5qSFgJ1JKg/o5fY5a3ukTK/AISXyKrc7XLO2fOVZS+aMJLFNCDgJRoIEqEogTemQ== X-Received: by 2002:a50:ed89:: with SMTP id h9mr10209322edr.217.1567784084108; Fri, 06 Sep 2019 08:34:44 -0700 (PDT) Received: from localhost.localdomain.com ([188.204.2.113]) by smtp.gmail.com with ESMTPSA id c22sm995218eds.30.2019.09.06.08.34.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Sep 2019 08:34:43 -0700 (PDT) From: Pascal van Leeuwen X-Google-Original-From: Pascal van Leeuwen To: linux-crypto@vger.kernel.org Cc: antoine.tenart@bootlin.com, herbert@gondor.apana.org.au, davem@davemloft.net, Pascal van Leeuwen Subject: [PATCH 6/6] crypto: inside-secure - Probe transform record cache RAM sizes Date: Fri, 6 Sep 2019 16:31:53 +0200 Message-Id: <1567780313-1579-7-git-send-email-pvanleeuwen@verimatrix.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1567780313-1579-1-git-send-email-pvanleeuwen@verimatrix.com> References: <1567780313-1579-1-git-send-email-pvanleeuwen@verimatrix.com> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org This patch actually probes the transform record cache data and administration RAM sizes, instead of making assumptions, and then configures the TRC based on the actually probed values. This allows the driver to work with EIP197 HW that has TRC RAM sizes different from those of the Marvell EIP197B/D variants. Signed-off-by: Pascal van Leeuwen --- drivers/crypto/inside-secure/safexcel.c | 229 ++++++++++++++++++++++++++------ drivers/crypto/inside-secure/safexcel.h | 21 +-- 2 files changed, 200 insertions(+), 50 deletions(-) diff --git a/drivers/crypto/inside-secure/safexcel.c b/drivers/crypto/inside-secure/safexcel.c index a607786..a34bf8c 100644 --- a/drivers/crypto/inside-secure/safexcel.c +++ b/drivers/crypto/inside-secure/safexcel.c @@ -28,63 +28,205 @@ module_param(max_rings, uint, 0644); MODULE_PARM_DESC(max_rings, "Maximum number of rings to use."); -static void eip197_trc_cache_init(struct safexcel_crypto_priv *priv) +static void eip197_trc_cache_setupvirt(struct safexcel_crypto_priv *priv) { - u32 val, htable_offset; - int i, cs_rc_max, cs_ht_wc, cs_trc_rec_wc, cs_trc_lg_rec_wc; - - if (priv->version == EIP197D_MRVL) { - cs_rc_max = EIP197D_CS_RC_MAX; - cs_ht_wc = EIP197D_CS_HT_WC; - cs_trc_rec_wc = EIP197D_CS_TRC_REC_WC; - cs_trc_lg_rec_wc = EIP197D_CS_TRC_LG_REC_WC; - } else { - /* Default to minimum "safe" settings */ - cs_rc_max = EIP197B_CS_RC_MAX; - cs_ht_wc = EIP197B_CS_HT_WC; - cs_trc_rec_wc = EIP197B_CS_TRC_REC_WC; - cs_trc_lg_rec_wc = EIP197B_CS_TRC_LG_REC_WC; + int i; + + /* + * Map all interfaces/rings to register index 0 + * so they can share contexts. Without this, the EIP197 will + * assume each interface/ring to be in its own memory domain + * i.e. have its own subset of UNIQUE memory addresses. + * Which would cause records with the SAME memory address to + * use DIFFERENT cache buffers, causing both poor cache utilization + * AND serious coherence/invalidation issues. + */ + for (i = 0; i < 4; i++) + writel(0, priv->base + EIP197_FLUE_IFC_LUT(i)); + + /* + * Initialize other virtualization regs for cache + * These may not be in their reset state ... + */ + for (i = 0; i < priv->config.rings; i++) { + writel(0, priv->base + EIP197_FLUE_CACHEBASE_LO(i)); + writel(0, priv->base + EIP197_FLUE_CACHEBASE_HI(i)); + writel(EIP197_FLUE_CONFIG_MAGIC, + priv->base + EIP197_FLUE_CONFIG(i)); } + writel(0, priv->base + EIP197_FLUE_OFFSETS); + writel(0, priv->base + EIP197_FLUE_ARC4_OFFSET); +} - /* Enable the record cache memory access */ - val = readl(priv->base + EIP197_CS_RAM_CTRL); - val &= ~EIP197_TRC_ENABLE_MASK; - val |= EIP197_TRC_ENABLE_0; - writel(val, priv->base + EIP197_CS_RAM_CTRL); +static void eip197_trc_cache_banksel(struct safexcel_crypto_priv *priv, + u32 addrmid, int *actbank) +{ + u32 val; + int curbank; + + curbank = addrmid >> 16; + if (curbank != *actbank) { + val = readl(priv->base + EIP197_CS_RAM_CTRL); + val = (val & ~EIP197_CS_BANKSEL_MASK) | + (curbank << EIP197_CS_BANKSEL_OFS); + writel(val, priv->base + EIP197_CS_RAM_CTRL); + *actbank = curbank; + } +} - /* Clear all ECC errors */ - writel(0, priv->base + EIP197_TRC_ECCCTRL); +static u32 eip197_trc_cache_probe(struct safexcel_crypto_priv *priv, + int maxbanks, u32 probemask) +{ + u32 val, addrhi, addrlo, addrmid; + int actbank; /* - * Make sure the cache memory is accessible by taking record cache into - * reset. + * And probe the actual size of the physically attached cache data RAM + * Using a binary subdivision algorithm downto 32 byte cache lines. */ - val = readl(priv->base + EIP197_TRC_PARAMS); - val |= EIP197_TRC_PARAMS_SW_RESET; - val &= ~EIP197_TRC_PARAMS_DATA_ACCESS; - writel(val, priv->base + EIP197_TRC_PARAMS); + addrhi = 1 << (16 + maxbanks); + addrlo = 0; + actbank = min(maxbanks - 1, 0); + while ((addrhi - addrlo) > 32) { + /* write marker to lowest address in top half */ + addrmid = (addrhi + addrlo) >> 1; + eip197_trc_cache_banksel(priv, addrmid, &actbank); + writel((addrmid | (addrlo << 16)) & probemask, + priv->base + EIP197_CLASSIFICATION_RAMS + + (addrmid & 0xffff)); + + /* write marker to lowest address in bottom half */ + eip197_trc_cache_banksel(priv, addrlo, &actbank); + writel((addrlo | (addrhi << 16)) & probemask, + priv->base + EIP197_CLASSIFICATION_RAMS + + (addrlo & 0xffff)); + + /* read back marker from top half */ + eip197_trc_cache_banksel(priv, addrmid, &actbank); + val = readl(priv->base + EIP197_CLASSIFICATION_RAMS + + (addrmid & 0xffff)); + + if (val == ((addrmid | (addrlo << 16)) & probemask)) { + /* read back correct, continue with top half */ + addrlo = addrmid; + } else { + /* not read back correct, continue with bottom half */ + addrhi = addrmid; + } + } + return addrhi; +} + +static void eip197_trc_cache_clear(struct safexcel_crypto_priv *priv, + int cs_rc_max, int cs_ht_wc) +{ + int i; + u32 htable_offset, val, offset; - /* Clear all records */ + /* Clear all records in administration RAM */ for (i = 0; i < cs_rc_max; i++) { - u32 val, offset = EIP197_CLASSIFICATION_RAMS + i * EIP197_CS_RC_SIZE; + offset = EIP197_CLASSIFICATION_RAMS + i * EIP197_CS_RC_SIZE; writel(EIP197_CS_RC_NEXT(EIP197_RC_NULL) | EIP197_CS_RC_PREV(EIP197_RC_NULL), priv->base + offset); - val = EIP197_CS_RC_NEXT(i+1) | EIP197_CS_RC_PREV(i-1); + val = EIP197_CS_RC_NEXT(i + 1) | EIP197_CS_RC_PREV(i - 1); if (i == 0) val |= EIP197_CS_RC_PREV(EIP197_RC_NULL); else if (i == cs_rc_max - 1) val |= EIP197_CS_RC_NEXT(EIP197_RC_NULL); - writel(val, priv->base + offset + sizeof(u32)); + writel(val, priv->base + offset + 4); + /* must also initialize the address key due to ECC! */ + writel(0, priv->base + offset + 8); + writel(0, priv->base + offset + 12); } /* Clear the hash table entries */ htable_offset = cs_rc_max * EIP197_CS_RC_SIZE; for (i = 0; i < cs_ht_wc; i++) writel(GENMASK(29, 0), - priv->base + EIP197_CLASSIFICATION_RAMS + htable_offset + i * sizeof(u32)); + priv->base + EIP197_CLASSIFICATION_RAMS + + htable_offset + i * sizeof(u32)); +} + +static void eip197_trc_cache_init(struct safexcel_crypto_priv *priv) +{ + u32 val, dsize, asize; + int cs_rc_max, cs_ht_wc, cs_trc_rec_wc, cs_trc_lg_rec_wc; + int cs_rc_abs_max, cs_ht_sz; + int maxbanks; + + /* Setup (dummy) virtualization for cache */ + eip197_trc_cache_setupvirt(priv); + + /* + * Enable the record cache memory access and + * probe the bank select width + */ + val = readl(priv->base + EIP197_CS_RAM_CTRL); + val &= ~EIP197_TRC_ENABLE_MASK; + val |= EIP197_TRC_ENABLE_0 | EIP197_CS_BANKSEL_MASK; + writel(val, priv->base + EIP197_CS_RAM_CTRL); + val = readl(priv->base + EIP197_CS_RAM_CTRL); + maxbanks = ((val&EIP197_CS_BANKSEL_MASK)>>EIP197_CS_BANKSEL_OFS) + 1; + + /* Clear all ECC errors */ + writel(0, priv->base + EIP197_TRC_ECCCTRL); + + /* + * Make sure the cache memory is accessible by taking record cache into + * reset. Need data memory access here, not admin access. + */ + val = readl(priv->base + EIP197_TRC_PARAMS); + val |= EIP197_TRC_PARAMS_SW_RESET | EIP197_TRC_PARAMS_DATA_ACCESS; + writel(val, priv->base + EIP197_TRC_PARAMS); + + /* Probed data RAM size in bytes */ + dsize = eip197_trc_cache_probe(priv, maxbanks, 0xffffffff); + + /* + * Now probe the administration RAM size pretty much the same way + * Except that only the lower 30 bits are writable and we don't need + * bank selects + */ + val = readl(priv->base + EIP197_TRC_PARAMS); + /* admin access now */ + val &= ~(EIP197_TRC_PARAMS_DATA_ACCESS | EIP197_CS_BANKSEL_MASK); + writel(val, priv->base + EIP197_TRC_PARAMS); + + /* Probed admin RAM size in admin words */ + asize = eip197_trc_cache_probe(priv, 0, 0xbfffffff) >> 4; + + /* Clear any ECC errors detected while probing! */ + writel(0, priv->base + EIP197_TRC_ECCCTRL); + + /* + * Determine optimal configuration from RAM sizes + * Note that we assume that the physical RAM configuration is sane + * Therefore, we don't do any parameter error checking here ... + */ + + /* For now, just use a single record format covering everything */ + cs_trc_rec_wc = EIP197_CS_TRC_REC_WC; + cs_trc_lg_rec_wc = EIP197_CS_TRC_REC_WC; + + /* + * Step #1: How many records will physically fit? + * Hard upper limit is 1023! + */ + cs_rc_abs_max = min_t(uint, ((dsize >> 2) / cs_trc_lg_rec_wc), 1023); + /* Step #2: Need at least 2 words in the admin RAM per record */ + cs_rc_max = min_t(uint, cs_rc_abs_max, (asize >> 1)); + /* Step #3: Determine log2 of hash table size */ + cs_ht_sz = __fls(asize - cs_rc_max) - 2; + /* Step #4: determine current size of hash table in dwords */ + cs_ht_wc = 16<> 4)); + + /* Clear the cache RAMs */ + eip197_trc_cache_clear(priv, cs_rc_max, cs_ht_wc); /* Disable the record cache memory access */ val = readl(priv->base + EIP197_CS_RAM_CTRL); @@ -104,8 +246,11 @@ static void eip197_trc_cache_init(struct safexcel_crypto_priv *priv) /* Configure the record cache #2 */ val = EIP197_TRC_PARAMS_RC_SZ_LARGE(cs_trc_lg_rec_wc) | EIP197_TRC_PARAMS_BLK_TIMER_SPEED(1) | - EIP197_TRC_PARAMS_HTABLE_SZ(2); + EIP197_TRC_PARAMS_HTABLE_SZ(cs_ht_sz); writel(val, priv->base + EIP197_TRC_PARAMS); + + dev_info(priv->dev, "TRC init: %dd,%da (%dr,%dh)\n", + dsize, asize, cs_rc_max, cs_ht_wc + cs_ht_wc); } static void eip197_init_firmware(struct safexcel_crypto_priv *priv) @@ -129,7 +274,7 @@ static void eip197_init_firmware(struct safexcel_crypto_priv *priv) /* clear the scratchpad RAM using 32 bit writes only */ for (i = 0; i < EIP197_NUM_OF_SCRATCH_BLOCKS; i++) writel(0, EIP197_PE(priv) + - EIP197_PE_ICE_SCRATCH_RAM(pe) + (i<<2)); + EIP197_PE_ICE_SCRATCH_RAM(pe) + (i << 2)); /* Reset the IFPP engine to make its program mem accessible */ writel(EIP197_PE_ICE_x_CTRL_SW_RESET | @@ -309,7 +454,7 @@ static int eip197_load_firmwares(struct safexcel_crypto_priv *priv) static int safexcel_hw_setup_cdesc_rings(struct safexcel_crypto_priv *priv) { - u32 hdw, cd_size_rnd, val; + u32 cd_size_rnd, val; int i, cd_fetch_cnt; cd_size_rnd = (priv->config.cd_size + @@ -337,7 +482,8 @@ static int safexcel_hw_setup_cdesc_rings(struct safexcel_crypto_priv *priv) writel(EIP197_xDR_DESC_MODE_64BIT | (priv->config.cd_offset << 16) | priv->config.cd_size, EIP197_HIA_CDR(priv, i) + EIP197_HIA_xDR_DESC_SIZE); - writel(((cd_fetch_cnt * (cd_size_rnd << hdw)) << 16) | + writel(((cd_fetch_cnt * + (cd_size_rnd << priv->hwconfig.hwdataw)) << 16) | (cd_fetch_cnt * priv->config.cd_offset), EIP197_HIA_CDR(priv, i) + EIP197_HIA_xDR_CFG); @@ -356,12 +502,12 @@ static int safexcel_hw_setup_cdesc_rings(struct safexcel_crypto_priv *priv) static int safexcel_hw_setup_rdesc_rings(struct safexcel_crypto_priv *priv) { - u32 hdw, rd_size_rnd, val; + u32 rd_size_rnd, val; int i, rd_fetch_cnt; /* determine number of RD's we can fetch into the FIFO as one block */ rd_size_rnd = (EIP197_RD64_FETCH_SIZE + - BIT(priv->hwconfig.hwdataw) - 1) >> + (BIT(priv->hwconfig.hwdataw) - 1)) >> priv->hwconfig.hwdataw; if (priv->flags & SAFEXCEL_HW_EIP197) { /* EIP197: try to fetch enough in 1 go to keep all pipes busy */ @@ -371,7 +517,7 @@ static int safexcel_hw_setup_rdesc_rings(struct safexcel_crypto_priv *priv) } else { /* for the EIP97, just fetch all that fits minus 1 */ rd_fetch_cnt = ((1 << priv->hwconfig.hwrfsize) / - rd_size_rnd) - 1; + rd_size_rnd) - 1; } for (i = 0; i < priv->config.rings; i++) { @@ -385,7 +531,8 @@ static int safexcel_hw_setup_rdesc_rings(struct safexcel_crypto_priv *priv) priv->config.rd_size, EIP197_HIA_RDR(priv, i) + EIP197_HIA_xDR_DESC_SIZE); - writel(((rd_fetch_cnt * (rd_size_rnd << hdw)) << 16) | + writel(((rd_fetch_cnt * + (rd_size_rnd << priv->hwconfig.hwdataw)) << 16) | (rd_fetch_cnt * priv->config.rd_offset), EIP197_HIA_RDR(priv, i) + EIP197_HIA_xDR_CFG); diff --git a/drivers/crypto/inside-secure/safexcel.h b/drivers/crypto/inside-secure/safexcel.h index 19049f1..6ddc6d1 100644 --- a/drivers/crypto/inside-secure/safexcel.h +++ b/drivers/crypto/inside-secure/safexcel.h @@ -178,6 +178,12 @@ #define EIP197_TRC_ECCADMINSTAT 0xf0838 #define EIP197_TRC_ECCDATASTAT 0xf083c #define EIP197_TRC_ECCDATA 0xf0840 +#define EIP197_FLUE_CACHEBASE_LO(n) (0xf6000 + (32 * (n))) +#define EIP197_FLUE_CACHEBASE_HI(n) (0xf6004 + (32 * (n))) +#define EIP197_FLUE_CONFIG(n) (0xf6010 + (32 * (n))) +#define EIP197_FLUE_OFFSETS 0xf6808 +#define EIP197_FLUE_ARC4_OFFSET 0xf680c +#define EIP197_FLUE_IFC_LUT(n) (0xf6820 + (4 * (n))) #define EIP197_CS_RAM_CTRL 0xf7ff0 /* EIP197_HIA_xDR_DESC_SIZE */ @@ -321,6 +327,9 @@ #define EIP197_ADDRESS_MODE BIT(8) #define EIP197_CONTROL_MODE BIT(9) +/* EIP197_FLUE_CONFIG */ +#define EIP197_FLUE_CONFIG_MAGIC 0xc7000004 + /* Context Control */ struct safexcel_context_record { u32 control0; @@ -396,6 +405,8 @@ struct safexcel_context_record { #define EIP197_TRC_ENABLE_1 BIT(5) #define EIP197_TRC_ENABLE_2 BIT(6) #define EIP197_TRC_ENABLE_MASK GENMASK(6, 4) +#define EIP197_CS_BANKSEL_MASK GENMASK(14, 12) +#define EIP197_CS_BANKSEL_OFS 12 /* EIP197_TRC_PARAMS */ #define EIP197_TRC_PARAMS_SW_RESET BIT(0) @@ -413,19 +424,11 @@ struct safexcel_context_record { #define EIP197_TRC_PARAMS2_RC_SZ_SMALL(n) ((n) << 18) /* Cache helpers */ -#define EIP197B_CS_RC_MAX 52 -#define EIP197D_CS_RC_MAX 96 +#define EIP197_CS_TRC_REC_WC 64 #define EIP197_CS_RC_SIZE (4 * sizeof(u32)) #define EIP197_CS_RC_NEXT(x) (x) #define EIP197_CS_RC_PREV(x) ((x) << 10) #define EIP197_RC_NULL 0x3ff -#define EIP197B_CS_TRC_REC_WC 59 -#define EIP197D_CS_TRC_REC_WC 64 -#define EIP197B_CS_TRC_LG_REC_WC 73 -#define EIP197D_CS_TRC_LG_REC_WC 80 -#define EIP197B_CS_HT_WC 64 -#define EIP197D_CS_HT_WC 256 - /* Result data */ struct result_data_desc {