From patchwork Mon Feb 11 11:46:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Horia Geanta X-Patchwork-Id: 10805551 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 20B38746 for ; Mon, 11 Feb 2019 11:46:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 109CF29BFC for ; Mon, 11 Feb 2019 11:46:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0529C29C02; Mon, 11 Feb 2019 11:46:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7C2A229BFC for ; Mon, 11 Feb 2019 11:46:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726041AbfBKLqd (ORCPT ); Mon, 11 Feb 2019 06:46:33 -0500 Received: from inva021.nxp.com ([92.121.34.21]:52530 "EHLO inva021.nxp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726883AbfBKLqd (ORCPT ); Mon, 11 Feb 2019 06:46:33 -0500 Received: from inva021.nxp.com (localhost [127.0.0.1]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 5224F2000CC; Mon, 11 Feb 2019 12:46:30 +0100 (CET) Received: from inva024.eu-rdc02.nxp.com (inva024.eu-rdc02.nxp.com [134.27.226.22]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 3FD25200009; Mon, 11 Feb 2019 12:46:30 +0100 (CET) Received: from enigma.ea.freescale.net (enigma.ea.freescale.net [10.171.81.110]) by inva024.eu-rdc02.nxp.com (Postfix) with ESMTP id E214B205FA; Mon, 11 Feb 2019 12:46:29 +0100 (CET) From: =?utf-8?q?Horia_Geant=C4=83?= To: Herbert Xu Cc: "David S. Miller" , Aymen Sghaier , linux-crypto@vger.kernel.org, Leo Li , Roy Pledge , Vakul Garg Subject: [PATCH 1/3] crypto: caam/qi2 - use affine DPIOs Date: Mon, 11 Feb 2019 13:46:12 +0200 Message-Id: <20190211114614.24748-2-horia.geanta@nxp.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20190211114614.24748-1-horia.geanta@nxp.com> References: <20190211114614.24748-1-horia.geanta@nxp.com> MIME-Version: 1.0 X-Virus-Scanned: ClamAV using ClamSMTP Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Driver was relying on an older DPIO API, which provided a CPU-affine DPIO in case it was called with preemption disabled. Since this is no longer the case, save the CPU-affine DPIO in per-cpu private structure during setup and further use it on the hot path. Note that preemption is no longer disabled while trying to enqueue an FD. Thus it might be possible to run the enqueue on a different CPU (due to migration, when in process context), however this wouldn't be a functionality issue. Since we allow for all cores to enqueue, we take care of data structures setup to handle the case when number of (Rx, Tx) queue pairs is smaller than number of cores. Signed-off-by: Horia Geantă --- drivers/crypto/caam/caamalg_qi2.c | 52 ++++++++++++++++++++++----------------- drivers/crypto/caam/caamalg_qi2.h | 2 ++ 2 files changed, 31 insertions(+), 23 deletions(-) diff --git a/drivers/crypto/caam/caamalg_qi2.c b/drivers/crypto/caam/caamalg_qi2.c index 185365a35772..ce27c88e750f 100644 --- a/drivers/crypto/caam/caamalg_qi2.c +++ b/drivers/crypto/caam/caamalg_qi2.c @@ -4502,7 +4502,8 @@ static int __cold dpaa2_dpseci_dpio_setup(struct dpaa2_caam_priv *priv) nctx->cb = dpaa2_caam_fqdan_cb; /* Register notification callbacks */ - err = dpaa2_io_service_register(NULL, nctx, dev); + ppriv->dpio = dpaa2_io_service_select(cpu); + err = dpaa2_io_service_register(ppriv->dpio, nctx, dev); if (unlikely(err)) { dev_dbg(dev, "No affine DPIO for cpu %d\n", cpu); nctx->cb = NULL; @@ -4535,7 +4536,7 @@ static int __cold dpaa2_dpseci_dpio_setup(struct dpaa2_caam_priv *priv) ppriv = per_cpu_ptr(priv->ppriv, cpu); if (!ppriv->nctx.cb) break; - dpaa2_io_service_deregister(NULL, &ppriv->nctx, dev); + dpaa2_io_service_deregister(ppriv->dpio, &ppriv->nctx, dev); } for_each_online_cpu(cpu) { @@ -4555,7 +4556,8 @@ static void __cold dpaa2_dpseci_dpio_free(struct dpaa2_caam_priv *priv) for_each_online_cpu(cpu) { ppriv = per_cpu_ptr(priv->ppriv, cpu); - dpaa2_io_service_deregister(NULL, &ppriv->nctx, priv->dev); + dpaa2_io_service_deregister(ppriv->dpio, &ppriv->nctx, + priv->dev); dpaa2_io_store_destroy(ppriv->store); if (++i == priv->num_pairs) @@ -4653,7 +4655,7 @@ static int dpaa2_caam_pull_fq(struct dpaa2_caam_priv_per_cpu *ppriv) /* Retry while portal is busy */ do { - err = dpaa2_io_service_pull_fq(NULL, ppriv->rsp_fqid, + err = dpaa2_io_service_pull_fq(ppriv->dpio, ppriv->rsp_fqid, ppriv->store); } while (err == -EBUSY); @@ -4721,7 +4723,7 @@ static int dpaa2_dpseci_poll(struct napi_struct *napi, int budget) if (cleaned < budget) { napi_complete_done(napi, cleaned); - err = dpaa2_io_service_rearm(NULL, &ppriv->nctx); + err = dpaa2_io_service_rearm(ppriv->dpio, &ppriv->nctx); if (unlikely(err)) dev_err(priv->dev, "Notification rearm failed: %d\n", err); @@ -4862,21 +4864,31 @@ static int __cold dpaa2_dpseci_setup(struct fsl_mc_device *ls_dev) i = 0; for_each_online_cpu(cpu) { - dev_dbg(dev, "pair %d: rx queue %d, tx queue %d\n", i, - priv->rx_queue_attr[i].fqid, - priv->tx_queue_attr[i].fqid); + u8 j; + + j = i % priv->num_pairs; ppriv = per_cpu_ptr(priv->ppriv, cpu); - ppriv->req_fqid = priv->tx_queue_attr[i].fqid; - ppriv->rsp_fqid = priv->rx_queue_attr[i].fqid; - ppriv->prio = i; + ppriv->req_fqid = priv->tx_queue_attr[j].fqid; + + /* + * Allow all cores to enqueue, while only some of them + * will take part in dequeuing. + */ + if (++i > priv->num_pairs) + continue; + + ppriv->rsp_fqid = priv->rx_queue_attr[j].fqid; + ppriv->prio = j; + + dev_dbg(dev, "pair %d: rx queue %d, tx queue %d\n", j, + priv->rx_queue_attr[j].fqid, + priv->tx_queue_attr[j].fqid); ppriv->net_dev.dev = *dev; INIT_LIST_HEAD(&ppriv->net_dev.napi_list); netif_napi_add(&ppriv->net_dev, &ppriv->napi, dpaa2_dpseci_poll, DPAA2_CAAM_NAPI_WEIGHT); - if (++i == priv->num_pairs) - break; } return 0; @@ -5228,7 +5240,8 @@ int dpaa2_caam_enqueue(struct device *dev, struct caam_request *req) { struct dpaa2_fd fd; struct dpaa2_caam_priv *priv = dev_get_drvdata(dev); - int err = 0, i, id; + struct dpaa2_caam_priv_per_cpu *ppriv; + int err = 0, i; if (IS_ERR(req)) return PTR_ERR(req); @@ -5258,20 +5271,13 @@ int dpaa2_caam_enqueue(struct device *dev, struct caam_request *req) dpaa2_fd_set_len(&fd, dpaa2_fl_get_len(&req->fd_flt[1])); dpaa2_fd_set_flc(&fd, req->flc_dma); - /* - * There is no guarantee that preemption is disabled here, - * thus take action. - */ - preempt_disable(); - id = smp_processor_id() % priv->dpseci_attr.num_tx_queues; + ppriv = this_cpu_ptr(priv->ppriv); for (i = 0; i < (priv->dpseci_attr.num_tx_queues << 1); i++) { - err = dpaa2_io_service_enqueue_fq(NULL, - priv->tx_queue_attr[id].fqid, + err = dpaa2_io_service_enqueue_fq(ppriv->dpio, ppriv->req_fqid, &fd); if (err != -EBUSY) break; } - preempt_enable(); if (unlikely(err)) { dev_err(dev, "Error enqueuing frame: %d\n", err); diff --git a/drivers/crypto/caam/caamalg_qi2.h b/drivers/crypto/caam/caamalg_qi2.h index 9823bdefd029..20890780fb82 100644 --- a/drivers/crypto/caam/caamalg_qi2.h +++ b/drivers/crypto/caam/caamalg_qi2.h @@ -76,6 +76,7 @@ struct dpaa2_caam_priv { * @nctx: notification context of response FQ * @store: where dequeued frames are stored * @priv: backpointer to dpaa2_caam_priv + * @dpio: portal used for data path operations */ struct dpaa2_caam_priv_per_cpu { struct napi_struct napi; @@ -86,6 +87,7 @@ struct dpaa2_caam_priv_per_cpu { struct dpaa2_io_notification_ctx nctx; struct dpaa2_io_store *store; struct dpaa2_caam_priv *priv; + struct dpaa2_io *dpio; }; /* From patchwork Mon Feb 11 11:46:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Horia Geanta X-Patchwork-Id: 10805547 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A0F87922 for ; Mon, 11 Feb 2019 11:46:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 900DA29BFC for ; Mon, 11 Feb 2019 11:46:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8468C29C02; Mon, 11 Feb 2019 11:46:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0AC5529BFE for ; Mon, 11 Feb 2019 11:46:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726986AbfBKLqc (ORCPT ); Mon, 11 Feb 2019 06:46:32 -0500 Received: from inva021.nxp.com ([92.121.34.21]:52544 "EHLO inva021.nxp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726975AbfBKLqc (ORCPT ); Mon, 11 Feb 2019 06:46:32 -0500 Received: from inva021.nxp.com (localhost [127.0.0.1]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id BED1D200178; Mon, 11 Feb 2019 12:46:30 +0100 (CET) Received: from inva024.eu-rdc02.nxp.com (inva024.eu-rdc02.nxp.com [134.27.226.22]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id A9ED4200009; Mon, 11 Feb 2019 12:46:30 +0100 (CET) Received: from enigma.ea.freescale.net (enigma.ea.freescale.net [10.171.81.110]) by inva024.eu-rdc02.nxp.com (Postfix) with ESMTP id 4E042205FA; Mon, 11 Feb 2019 12:46:30 +0100 (CET) From: =?utf-8?q?Horia_Geant=C4=83?= To: Herbert Xu Cc: "David S. Miller" , Aymen Sghaier , linux-crypto@vger.kernel.org, Leo Li , Roy Pledge , Vakul Garg Subject: [PATCH 2/3] crypto: caam/qi2 - rate-limit enqueue failure prints Date: Mon, 11 Feb 2019 13:46:13 +0200 Message-Id: <20190211114614.24748-3-horia.geanta@nxp.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20190211114614.24748-1-horia.geanta@nxp.com> References: <20190211114614.24748-1-horia.geanta@nxp.com> MIME-Version: 1.0 X-Virus-Scanned: ClamAV using ClamSMTP Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Avoid console being flooded with prints in case HW is too busy to accept new enqueue requests. Signed-off-by: Horia Geantă --- drivers/crypto/caam/caamalg_qi2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/crypto/caam/caamalg_qi2.c b/drivers/crypto/caam/caamalg_qi2.c index ce27c88e750f..6a6a508f5fd2 100644 --- a/drivers/crypto/caam/caamalg_qi2.c +++ b/drivers/crypto/caam/caamalg_qi2.c @@ -5280,7 +5280,7 @@ int dpaa2_caam_enqueue(struct device *dev, struct caam_request *req) } if (unlikely(err)) { - dev_err(dev, "Error enqueuing frame: %d\n", err); + dev_err_ratelimited(dev, "Error enqueuing frame: %d\n", err); goto err_out; } From patchwork Mon Feb 11 11:46:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Horia Geanta X-Patchwork-Id: 10805549 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E207C184E for ; Mon, 11 Feb 2019 11:46:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D1FBC29BFC for ; Mon, 11 Feb 2019 11:46:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C6BD229C02; Mon, 11 Feb 2019 11:46:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 849BE29BFE for ; Mon, 11 Feb 2019 11:46:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726975AbfBKLqc (ORCPT ); Mon, 11 Feb 2019 06:46:32 -0500 Received: from inva020.nxp.com ([92.121.34.13]:46456 "EHLO inva020.nxp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726041AbfBKLqc (ORCPT ); Mon, 11 Feb 2019 06:46:32 -0500 Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 2B5201A034B; Mon, 11 Feb 2019 12:46:31 +0100 (CET) Received: from inva024.eu-rdc02.nxp.com (inva024.eu-rdc02.nxp.com [134.27.226.22]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 1F3841A01B7; Mon, 11 Feb 2019 12:46:31 +0100 (CET) Received: from enigma.ea.freescale.net (enigma.ea.freescale.net [10.171.81.110]) by inva024.eu-rdc02.nxp.com (Postfix) with ESMTP id B7E43205FA; Mon, 11 Feb 2019 12:46:30 +0100 (CET) From: =?utf-8?q?Horia_Geant=C4=83?= To: Herbert Xu Cc: "David S. Miller" , Aymen Sghaier , linux-crypto@vger.kernel.org, Leo Li , Roy Pledge , Vakul Garg Subject: [PATCH 3/3] crypto: caam/qi2 - relax busy polling while enqueuing FDs Date: Mon, 11 Feb 2019 13:46:14 +0200 Message-Id: <20190211114614.24748-4-horia.geanta@nxp.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20190211114614.24748-1-horia.geanta@nxp.com> References: <20190211114614.24748-1-horia.geanta@nxp.com> MIME-Version: 1.0 X-Virus-Scanned: ClamAV using ClamSMTP Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add cpu_relax() in the loop that tries to enqueue the FDs. Signed-off-by: Horia Geantă --- drivers/crypto/caam/caamalg_qi2.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/crypto/caam/caamalg_qi2.c b/drivers/crypto/caam/caamalg_qi2.c index 6a6a508f5fd2..c2c1abc68f81 100644 --- a/drivers/crypto/caam/caamalg_qi2.c +++ b/drivers/crypto/caam/caamalg_qi2.c @@ -5277,6 +5277,8 @@ int dpaa2_caam_enqueue(struct device *dev, struct caam_request *req) &fd); if (err != -EBUSY) break; + + cpu_relax(); } if (unlikely(err)) {