From patchwork Sat Jan 11 08:58:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Zaibo X-Patchwork-Id: 11328749 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 004391871 for ; Sat, 11 Jan 2020 09:02:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DD3712077C for ; Sat, 11 Jan 2020 09:02:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728027AbgAKJCO (ORCPT ); Sat, 11 Jan 2020 04:02:14 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:8697 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728417AbgAKJCO (ORCPT ); Sat, 11 Jan 2020 04:02:14 -0500 Received: from DGGEMS408-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id DEBE2DB374B07D05D707; Sat, 11 Jan 2020 17:02:11 +0800 (CST) Received: from localhost.localdomain (10.67.165.24) by DGGEMS408-HUB.china.huawei.com (10.3.19.208) with Microsoft SMTP Server id 14.3.439.0; Sat, 11 Jan 2020 17:02:04 +0800 From: Zaibo Xu To: , CC: , , , , , , , , , Subject: [PATCH 1/4] crypto: hisilicon - Bugfixed tfm leak Date: Sat, 11 Jan 2020 16:58:15 +0800 Message-ID: <1578733098-13863-2-git-send-email-xuzaibo@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1578733098-13863-1-git-send-email-xuzaibo@huawei.com> References: <1578733098-13863-1-git-send-email-xuzaibo@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.165.24] X-CFilter-Loop: Reflected Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org 1.Fixed the bug of software tfm leakage. 2.Update HW error log message. Signed-off-by: Zaibo Xu --- drivers/crypto/hisilicon/hpre/hpre_crypto.c | 7 ++++++- drivers/crypto/hisilicon/hpre/hpre_main.c | 24 ++++++++++++------------ 2 files changed, 18 insertions(+), 13 deletions(-) diff --git a/drivers/crypto/hisilicon/hpre/hpre_crypto.c b/drivers/crypto/hisilicon/hpre/hpre_crypto.c index 98f037e..d8b0152 100644 --- a/drivers/crypto/hisilicon/hpre/hpre_crypto.c +++ b/drivers/crypto/hisilicon/hpre/hpre_crypto.c @@ -1043,6 +1043,7 @@ static unsigned int hpre_rsa_max_size(struct crypto_akcipher *tfm) static int hpre_rsa_init_tfm(struct crypto_akcipher *tfm) { struct hpre_ctx *ctx = akcipher_tfm_ctx(tfm); + int ret; ctx->rsa.soft_tfm = crypto_alloc_akcipher("rsa-generic", 0, 0); if (IS_ERR(ctx->rsa.soft_tfm)) { @@ -1050,7 +1051,11 @@ static int hpre_rsa_init_tfm(struct crypto_akcipher *tfm) return PTR_ERR(ctx->rsa.soft_tfm); } - return hpre_ctx_init(ctx); + ret = hpre_ctx_init(ctx); + if (ret) + crypto_free_akcipher(ctx->rsa.soft_tfm); + + return ret; } static void hpre_rsa_exit_tfm(struct crypto_akcipher *tfm) diff --git a/drivers/crypto/hisilicon/hpre/hpre_main.c b/drivers/crypto/hisilicon/hpre/hpre_main.c index 711f5d1..753e43d 100644 --- a/drivers/crypto/hisilicon/hpre/hpre_main.c +++ b/drivers/crypto/hisilicon/hpre/hpre_main.c @@ -106,18 +106,18 @@ static const char * const hpre_debug_file_name[] = { }; static const struct hpre_hw_error hpre_hw_errors[] = { - { .int_msk = BIT(0), .msg = "hpre_ecc_1bitt_err" }, - { .int_msk = BIT(1), .msg = "hpre_ecc_2bit_err" }, - { .int_msk = BIT(2), .msg = "hpre_data_wr_err" }, - { .int_msk = BIT(3), .msg = "hpre_data_rd_err" }, - { .int_msk = BIT(4), .msg = "hpre_bd_rd_err" }, - { .int_msk = BIT(5), .msg = "hpre_ooo_2bit_ecc_err" }, - { .int_msk = BIT(6), .msg = "hpre_cltr1_htbt_tm_out_err" }, - { .int_msk = BIT(7), .msg = "hpre_cltr2_htbt_tm_out_err" }, - { .int_msk = BIT(8), .msg = "hpre_cltr3_htbt_tm_out_err" }, - { .int_msk = BIT(9), .msg = "hpre_cltr4_htbt_tm_out_err" }, - { .int_msk = GENMASK(15, 10), .msg = "hpre_ooo_rdrsp_err" }, - { .int_msk = GENMASK(21, 16), .msg = "hpre_ooo_wrrsp_err" }, + { .int_msk = BIT(0), .msg = "core_ecc_1bit_err_int_set" }, + { .int_msk = BIT(1), .msg = "core_ecc_2bit_err_int_set" }, + { .int_msk = BIT(2), .msg = "dat_wb_poison_int_set" }, + { .int_msk = BIT(3), .msg = "dat_rd_poison_int_set" }, + { .int_msk = BIT(4), .msg = "bd_rd_poison_int_set" }, + { .int_msk = BIT(5), .msg = "ooo_ecc_2bit_err_int_set" }, + { .int_msk = BIT(6), .msg = "cluster1_shb_timeout_int_set" }, + { .int_msk = BIT(7), .msg = "cluster2_shb_timeout_int_set" }, + { .int_msk = BIT(8), .msg = "cluster3_shb_timeout_int_set" }, + { .int_msk = BIT(9), .msg = "cluster4_shb_timeout_int_set" }, + { .int_msk = GENMASK(15, 10), .msg = "ooo_rdrsp_err_int_set" }, + { .int_msk = GENMASK(21, 16), .msg = "ooo_wrrsp_err_int_set" }, { /* sentinel */ } }; From patchwork Sat Jan 11 08:58:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Zaibo X-Patchwork-Id: 11328747 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D3BC118B8 for ; Sat, 11 Jan 2020 09:02:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BAFCB2077C for ; Sat, 11 Jan 2020 09:02:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728346AbgAKJCO (ORCPT ); Sat, 11 Jan 2020 04:02:14 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:8695 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728027AbgAKJCO (ORCPT ); Sat, 11 Jan 2020 04:02:14 -0500 Received: from DGGEMS408-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id D5226DF1A1B3DDEA7223; Sat, 11 Jan 2020 17:02:11 +0800 (CST) Received: from localhost.localdomain (10.67.165.24) by DGGEMS408-HUB.china.huawei.com (10.3.19.208) with Microsoft SMTP Server id 14.3.439.0; Sat, 11 Jan 2020 17:02:04 +0800 From: Zaibo Xu To: , CC: , , , , , , , , , Subject: [PATCH 2/4] crypto: hisilicon - Fixed some tiny bugs of HPRE Date: Sat, 11 Jan 2020 16:58:16 +0800 Message-ID: <1578733098-13863-3-git-send-email-xuzaibo@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1578733098-13863-1-git-send-email-xuzaibo@huawei.com> References: <1578733098-13863-1-git-send-email-xuzaibo@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.165.24] X-CFilter-Loop: Reflected Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org 1.Use memzero_explicit to clear key; 2.Fix some little endian writings; 3.Fix some other bugs and stuff of code style; Signed-off-by: Zaibo Xu --- drivers/crypto/hisilicon/hpre/hpre_crypto.c | 62 +++++++++++++---------------- drivers/crypto/hisilicon/hpre/hpre_main.c | 8 ++-- 2 files changed, 32 insertions(+), 38 deletions(-) diff --git a/drivers/crypto/hisilicon/hpre/hpre_crypto.c b/drivers/crypto/hisilicon/hpre/hpre_crypto.c index d8b0152..76540a1 100644 --- a/drivers/crypto/hisilicon/hpre/hpre_crypto.c +++ b/drivers/crypto/hisilicon/hpre/hpre_crypto.c @@ -174,8 +174,8 @@ static struct hisi_qp *hpre_get_qp_and_start(void) } static int hpre_get_data_dma_addr(struct hpre_asym_request *hpre_req, - struct scatterlist *data, unsigned int len, - int is_src, dma_addr_t *tmp) + struct scatterlist *data, unsigned int len, + int is_src, dma_addr_t *tmp) { struct hpre_ctx *ctx = hpre_req->ctx; struct device *dev = HPRE_DEV(ctx); @@ -199,8 +199,8 @@ static int hpre_get_data_dma_addr(struct hpre_asym_request *hpre_req, } static int hpre_prepare_dma_buf(struct hpre_asym_request *hpre_req, - struct scatterlist *data, unsigned int len, - int is_src, dma_addr_t *tmp) + struct scatterlist *data, unsigned int len, + int is_src, dma_addr_t *tmp) { struct hpre_ctx *ctx = hpre_req->ctx; struct device *dev = HPRE_DEV(ctx); @@ -226,12 +226,12 @@ static int hpre_prepare_dma_buf(struct hpre_asym_request *hpre_req, } static int hpre_hw_data_init(struct hpre_asym_request *hpre_req, - struct scatterlist *data, unsigned int len, - int is_src, int is_dh) + struct scatterlist *data, unsigned int len, + int is_src, int is_dh) { struct hpre_sqe *msg = &hpre_req->req; struct hpre_ctx *ctx = hpre_req->ctx; - dma_addr_t tmp; + dma_addr_t tmp = 0; int ret; /* when the data is dh's source, we should format it */ @@ -253,8 +253,9 @@ static int hpre_hw_data_init(struct hpre_asym_request *hpre_req, } static void hpre_hw_data_clr_all(struct hpre_ctx *ctx, - struct hpre_asym_request *req, - struct scatterlist *dst, struct scatterlist *src) + struct hpre_asym_request *req, + struct scatterlist *dst, + struct scatterlist *src) { struct device *dev = HPRE_DEV(ctx); struct hpre_sqe *sqe = &req->req; @@ -288,7 +289,7 @@ static void hpre_hw_data_clr_all(struct hpre_ctx *ctx, } static int hpre_alg_res_post_hf(struct hpre_ctx *ctx, struct hpre_sqe *sqe, - void **kreq) + void **kreq) { struct hpre_asym_request *req; int err, id, done; @@ -375,7 +376,7 @@ static void hpre_alg_cb(struct hisi_qp *qp, void *resp) struct hpre_ctx *ctx = qp->qp_ctx; struct hpre_sqe *sqe = resp; - ctx->req_list[sqe->tag]->cb(ctx, resp); + ctx->req_list[le16_to_cpu(sqe->tag)]->cb(ctx, resp); } static int hpre_ctx_init(struct hpre_ctx *ctx) @@ -454,9 +455,6 @@ static int hpre_dh_compute_value(struct kpp_request *req) int ctr = 0; int ret; - if (!ctx) - return -EINVAL; - ret = hpre_msg_request_set(ctx, req, false); if (ret) return ret; @@ -472,9 +470,9 @@ static int hpre_dh_compute_value(struct kpp_request *req) goto clear_all; if (ctx->crt_g2_mode && !req->src) - msg->dw0 |= HPRE_ALG_DH_G2; + msg->dw0 = cpu_to_le32(le32_to_cpu(msg->dw0) | HPRE_ALG_DH_G2); else - msg->dw0 |= HPRE_ALG_DH; + msg->dw0 = cpu_to_le32(le32_to_cpu(msg->dw0) | HPRE_ALG_DH); do { ret = hisi_qp_send(ctx->qp, msg); } while (ret == -EBUSY && ctr++ < HPRE_TRY_SEND_TIMES); @@ -520,12 +518,12 @@ static int hpre_dh_set_params(struct hpre_ctx *ctx, struct dh *params) return -EINVAL; if (hpre_is_dh_params_length_valid(params->p_size << - HPRE_BITS_2_BYTES_SHIFT)) + HPRE_BITS_2_BYTES_SHIFT)) return -EINVAL; sz = ctx->key_sz = params->p_size; ctx->dh.xa_p = dma_alloc_coherent(dev, sz << 1, - &ctx->dh.dma_xa_p, GFP_KERNEL); + &ctx->dh.dma_xa_p, GFP_KERNEL); if (!ctx->dh.xa_p) return -ENOMEM; @@ -559,13 +557,12 @@ static void hpre_dh_clear_ctx(struct hpre_ctx *ctx, bool is_clear_all) hisi_qm_stop_qp(ctx->qp); if (ctx->dh.g) { - memset(ctx->dh.g, 0, sz); dma_free_coherent(dev, sz, ctx->dh.g, ctx->dh.dma_g); ctx->dh.g = NULL; } if (ctx->dh.xa_p) { - memset(ctx->dh.xa_p, 0, sz); + memzero_explicit(ctx->dh.xa_p, sz); dma_free_coherent(dev, sz << 1, ctx->dh.xa_p, ctx->dh.dma_xa_p); ctx->dh.xa_p = NULL; @@ -661,9 +658,6 @@ static int hpre_rsa_enc(struct akcipher_request *req) int ctr = 0; int ret; - if (!ctx) - return -EINVAL; - /* For 512 and 1536 bits key size, use soft tfm instead */ if (ctx->key_sz == HPRE_RSA_512BITS_KSZ || ctx->key_sz == HPRE_RSA_1536BITS_KSZ) { @@ -680,7 +674,7 @@ static int hpre_rsa_enc(struct akcipher_request *req) if (ret) return ret; - msg->dw0 |= HPRE_ALG_NC_NCRT; + msg->dw0 |= cpu_to_le32(HPRE_ALG_NC_NCRT); msg->key = cpu_to_le64((u64)ctx->rsa.dma_pubkey); ret = hpre_hw_data_init(hpre_req, req->src, req->src_len, 1, 0); @@ -716,9 +710,6 @@ static int hpre_rsa_dec(struct akcipher_request *req) int ctr = 0; int ret; - if (!ctx) - return -EINVAL; - /* For 512 and 1536 bits key size, use soft tfm instead */ if (ctx->key_sz == HPRE_RSA_512BITS_KSZ || ctx->key_sz == HPRE_RSA_1536BITS_KSZ) { @@ -737,10 +728,12 @@ static int hpre_rsa_dec(struct akcipher_request *req) if (ctx->crt_g2_mode) { msg->key = cpu_to_le64((u64)ctx->rsa.dma_crt_prikey); - msg->dw0 |= HPRE_ALG_NC_CRT; + msg->dw0 = cpu_to_le32(le32_to_cpu(msg->dw0) | + HPRE_ALG_NC_CRT); } else { msg->key = cpu_to_le64((u64)ctx->rsa.dma_prikey); - msg->dw0 |= HPRE_ALG_NC_NCRT; + msg->dw0 = cpu_to_le32(le32_to_cpu(msg->dw0) | + HPRE_ALG_NC_NCRT); } ret = hpre_hw_data_init(hpre_req, req->src, req->src_len, 1, 0); @@ -811,10 +804,8 @@ static int hpre_rsa_set_e(struct hpre_ctx *ctx, const char *value, hpre_rsa_drop_leading_zeros(&ptr, &vlen); - if (!ctx->key_sz || !vlen || vlen > ctx->key_sz) { - ctx->rsa.pubkey = NULL; + if (!ctx->key_sz || !vlen || vlen > ctx->key_sz) return -EINVAL; - } memcpy(ctx->rsa.pubkey + ctx->key_sz - vlen, ptr, vlen); @@ -899,7 +890,7 @@ static int hpre_rsa_setkey_crt(struct hpre_ctx *ctx, struct rsa_key *rsa_key) free_key: offset = hlf_ksz * HPRE_CRT_PRMS; - memset(ctx->rsa.crt_prikey, 0, offset); + memzero_explicit(ctx->rsa.crt_prikey, offset); dma_free_coherent(dev, hlf_ksz * HPRE_CRT_PRMS, ctx->rsa.crt_prikey, ctx->rsa.dma_crt_prikey); ctx->rsa.crt_prikey = NULL; @@ -924,14 +915,15 @@ static void hpre_rsa_clear_ctx(struct hpre_ctx *ctx, bool is_clear_all) } if (ctx->rsa.crt_prikey) { - memset(ctx->rsa.crt_prikey, 0, half_key_sz * HPRE_CRT_PRMS); + memzero_explicit(ctx->rsa.crt_prikey, + half_key_sz * HPRE_CRT_PRMS); dma_free_coherent(dev, half_key_sz * HPRE_CRT_PRMS, ctx->rsa.crt_prikey, ctx->rsa.dma_crt_prikey); ctx->rsa.crt_prikey = NULL; } if (ctx->rsa.prikey) { - memset(ctx->rsa.prikey, 0, ctx->key_sz); + memzero_explicit(ctx->rsa.prikey, ctx->key_sz); dma_free_coherent(dev, ctx->key_sz << 1, ctx->rsa.prikey, ctx->rsa.dma_prikey); ctx->rsa.prikey = NULL; diff --git a/drivers/crypto/hisilicon/hpre/hpre_main.c b/drivers/crypto/hisilicon/hpre/hpre_main.c index 753e43d..401747d 100644 --- a/drivers/crypto/hisilicon/hpre/hpre_main.c +++ b/drivers/crypto/hisilicon/hpre/hpre_main.c @@ -73,7 +73,7 @@ #define HPRE_DBGFS_VAL_MAX_LEN 20 #define HPRE_PCI_DEVICE_ID 0xa258 #define HPRE_PCI_VF_DEVICE_ID 0xa259 -#define HPRE_ADDR(qm, offset) (qm->io_base + (offset)) +#define HPRE_ADDR(qm, offset) ((qm)->io_base + (offset)) #define HPRE_QM_USR_CFG_MASK 0xfffffffe #define HPRE_QM_AXI_CFG_MASK 0xffff #define HPRE_QM_VFG_AX_MASK 0xff @@ -490,7 +490,7 @@ static ssize_t hpre_ctrl_debug_read(struct file *filp, char __user *buf, return -EINVAL; } spin_unlock_irq(&file->lock); - ret = sprintf(tbuf, "%u\n", val); + ret = snprintf(tbuf, HPRE_DBGFS_VAL_MAX_LEN, "%u\n", val); return simple_read_from_buffer(buf, count, pos, tbuf, ret); } @@ -607,7 +607,9 @@ static int hpre_cluster_debugfs_init(struct hpre_debug *debug) int i, ret; for (i = 0; i < HPRE_CLUSTERS_NUM; i++) { - sprintf(buf, "cluster%d", i); + ret = snprintf(buf, HPRE_DBGFS_VAL_MAX_LEN, "cluster%d", i); + if (ret < 0) + return -EINVAL; tmp_d = debugfs_create_dir(buf, debug->debug_root); regset = devm_kzalloc(dev, sizeof(*regset), GFP_KERNEL); From patchwork Sat Jan 11 08:58:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Zaibo X-Patchwork-Id: 11328741 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 910E01580 for ; Sat, 11 Jan 2020 09:02:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7A02C20842 for ; Sat, 11 Jan 2020 09:02:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728275AbgAKJCN (ORCPT ); Sat, 11 Jan 2020 04:02:13 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:8696 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728031AbgAKJCN (ORCPT ); Sat, 11 Jan 2020 04:02:13 -0500 Received: from DGGEMS408-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id E292FFD7BEA9DCD671EC; Sat, 11 Jan 2020 17:02:11 +0800 (CST) Received: from localhost.localdomain (10.67.165.24) by DGGEMS408-HUB.china.huawei.com (10.3.19.208) with Microsoft SMTP Server id 14.3.439.0; Sat, 11 Jan 2020 17:02:04 +0800 From: Zaibo Xu To: , CC: , , , , , , , , , Subject: [PATCH 3/4] crypto: hisilicon - adjust hpre_crt_para_get Date: Sat, 11 Jan 2020 16:58:17 +0800 Message-ID: <1578733098-13863-4-git-send-email-xuzaibo@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1578733098-13863-1-git-send-email-xuzaibo@huawei.com> References: <1578733098-13863-1-git-send-email-xuzaibo@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.165.24] X-CFilter-Loop: Reflected Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Reorder the input parameters of hpre_crt_para_get to make it cleaner. Signed-off-by: Zaibo Xu --- drivers/crypto/hisilicon/hpre/hpre_crypto.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/drivers/crypto/hisilicon/hpre/hpre_crypto.c b/drivers/crypto/hisilicon/hpre/hpre_crypto.c index 76540a1..954134d 100644 --- a/drivers/crypto/hisilicon/hpre/hpre_crypto.c +++ b/drivers/crypto/hisilicon/hpre/hpre_crypto.c @@ -827,17 +827,17 @@ static int hpre_rsa_set_d(struct hpre_ctx *ctx, const char *value, return 0; } -static int hpre_crt_para_get(char *para, const char *raw, - unsigned int raw_sz, unsigned int para_size) +static int hpre_crt_para_get(char *para, size_t para_sz, + const char *raw, size_t raw_sz) { const char *ptr = raw; size_t len = raw_sz; hpre_rsa_drop_leading_zeros(&ptr, &len); - if (!len || len > para_size) + if (!len || len > para_sz) return -EINVAL; - memcpy(para + para_size - len, ptr, len); + memcpy(para + para_sz - len, ptr, len); return 0; } @@ -855,32 +855,32 @@ static int hpre_rsa_setkey_crt(struct hpre_ctx *ctx, struct rsa_key *rsa_key) if (!ctx->rsa.crt_prikey) return -ENOMEM; - ret = hpre_crt_para_get(ctx->rsa.crt_prikey, rsa_key->dq, - rsa_key->dq_sz, hlf_ksz); + ret = hpre_crt_para_get(ctx->rsa.crt_prikey, hlf_ksz, + rsa_key->dq, rsa_key->dq_sz); if (ret) goto free_key; offset = hlf_ksz; - ret = hpre_crt_para_get(ctx->rsa.crt_prikey + offset, rsa_key->dp, - rsa_key->dp_sz, hlf_ksz); + ret = hpre_crt_para_get(ctx->rsa.crt_prikey + offset, hlf_ksz, + rsa_key->dp, rsa_key->dp_sz); if (ret) goto free_key; offset = hlf_ksz * HPRE_CRT_Q; - ret = hpre_crt_para_get(ctx->rsa.crt_prikey + offset, - rsa_key->q, rsa_key->q_sz, hlf_ksz); + ret = hpre_crt_para_get(ctx->rsa.crt_prikey + offset, hlf_ksz, + rsa_key->q, rsa_key->q_sz); if (ret) goto free_key; offset = hlf_ksz * HPRE_CRT_P; - ret = hpre_crt_para_get(ctx->rsa.crt_prikey + offset, - rsa_key->p, rsa_key->p_sz, hlf_ksz); + ret = hpre_crt_para_get(ctx->rsa.crt_prikey + offset, hlf_ksz, + rsa_key->p, rsa_key->p_sz); if (ret) goto free_key; offset = hlf_ksz * HPRE_CRT_INV; - ret = hpre_crt_para_get(ctx->rsa.crt_prikey + offset, - rsa_key->qinv, rsa_key->qinv_sz, hlf_ksz); + ret = hpre_crt_para_get(ctx->rsa.crt_prikey + offset, hlf_ksz, + rsa_key->qinv, rsa_key->qinv_sz); if (ret) goto free_key; From patchwork Sat Jan 11 08:58:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Zaibo X-Patchwork-Id: 11328745 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B181A1398 for ; Sat, 11 Jan 2020 09:02:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9A85B2077C for ; Sat, 11 Jan 2020 09:02:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728031AbgAKJCO (ORCPT ); Sat, 11 Jan 2020 04:02:14 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:8694 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728346AbgAKJCN (ORCPT ); Sat, 11 Jan 2020 04:02:13 -0500 Received: from DGGEMS408-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id DA5FF426E62474ECB505; Sat, 11 Jan 2020 17:02:11 +0800 (CST) Received: from localhost.localdomain (10.67.165.24) by DGGEMS408-HUB.china.huawei.com (10.3.19.208) with Microsoft SMTP Server id 14.3.439.0; Sat, 11 Jan 2020 17:02:04 +0800 From: Zaibo Xu To: , CC: , , , , , , , , , Subject: [PATCH 4/4] crypto: hisilicon - add branch prediction macro Date: Sat, 11 Jan 2020 16:58:18 +0800 Message-ID: <1578733098-13863-5-git-send-email-xuzaibo@huawei.com> X-Mailer: git-send-email 2.8.1 In-Reply-To: <1578733098-13863-1-git-send-email-xuzaibo@huawei.com> References: <1578733098-13863-1-git-send-email-xuzaibo@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.165.24] X-CFilter-Loop: Reflected Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org This branch prediction macro on the hot path can improve small performance(about 2%) according to the test. Signed-off-by: Zaibo Xu --- drivers/crypto/hisilicon/hpre/hpre_crypto.c | 44 ++++++++++++++--------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/drivers/crypto/hisilicon/hpre/hpre_crypto.c b/drivers/crypto/hisilicon/hpre/hpre_crypto.c index 954134d..5d400d6 100644 --- a/drivers/crypto/hisilicon/hpre/hpre_crypto.c +++ b/drivers/crypto/hisilicon/hpre/hpre_crypto.c @@ -123,7 +123,7 @@ static int hpre_add_req_to_ctx(struct hpre_asym_request *hpre_req) ctx = hpre_req->ctx; id = hpre_alloc_req_id(ctx); - if (id < 0) + if (unlikely(id < 0)) return -EINVAL; ctx->req_list[id] = hpre_req; @@ -190,7 +190,7 @@ static int hpre_get_data_dma_addr(struct hpre_asym_request *hpre_req, } *tmp = dma_map_single(dev, sg_virt(data), len, dma_dir); - if (dma_mapping_error(dev, *tmp)) { + if (unlikely(dma_mapping_error(dev, *tmp))) { dev_err(dev, "dma map data err!\n"); return -ENOMEM; } @@ -208,11 +208,11 @@ static int hpre_prepare_dma_buf(struct hpre_asym_request *hpre_req, int shift; shift = ctx->key_sz - len; - if (shift < 0) + if (unlikely(shift < 0)) return -EINVAL; ptr = dma_alloc_coherent(dev, ctx->key_sz, tmp, GFP_KERNEL); - if (!ptr) + if (unlikely(!ptr)) return -ENOMEM; if (is_src) { @@ -241,7 +241,7 @@ static int hpre_hw_data_init(struct hpre_asym_request *hpre_req, else ret = hpre_prepare_dma_buf(hpre_req, data, len, is_src, &tmp); - if (ret) + if (unlikely(ret)) return ret; if (is_src) @@ -262,7 +262,7 @@ static void hpre_hw_data_clr_all(struct hpre_ctx *ctx, dma_addr_t tmp; tmp = le64_to_cpu(sqe->in); - if (!tmp) + if (unlikely(!tmp)) return; if (src) { @@ -275,7 +275,7 @@ static void hpre_hw_data_clr_all(struct hpre_ctx *ctx, } tmp = le64_to_cpu(sqe->out); - if (!tmp) + if (unlikely(!tmp)) return; if (req->dst) { @@ -309,7 +309,7 @@ static int hpre_alg_res_post_hf(struct hpre_ctx *ctx, struct hpre_sqe *sqe, done = (le32_to_cpu(sqe->dw0) >> HPRE_SQE_DONE_SHIFT) & HREE_SQE_DONE_MASK; - if (err == HPRE_NO_HW_ERR && done == HPRE_HW_TASK_DONE) + if (likely(err == HPRE_NO_HW_ERR && done == HPRE_HW_TASK_DONE)) return 0; return -EINVAL; @@ -456,17 +456,17 @@ static int hpre_dh_compute_value(struct kpp_request *req) int ret; ret = hpre_msg_request_set(ctx, req, false); - if (ret) + if (unlikely(ret)) return ret; if (req->src) { ret = hpre_hw_data_init(hpre_req, req->src, req->src_len, 1, 1); - if (ret) + if (unlikely(ret)) goto clear_all; } ret = hpre_hw_data_init(hpre_req, req->dst, req->dst_len, 0, 1); - if (ret) + if (unlikely(ret)) goto clear_all; if (ctx->crt_g2_mode && !req->src) @@ -478,7 +478,7 @@ static int hpre_dh_compute_value(struct kpp_request *req) } while (ret == -EBUSY && ctr++ < HPRE_TRY_SEND_TIMES); /* success */ - if (!ret) + if (likely(!ret)) return -EINPROGRESS; clear_all: @@ -667,22 +667,22 @@ static int hpre_rsa_enc(struct akcipher_request *req) return ret; } - if (!ctx->rsa.pubkey) + if (unlikely(!ctx->rsa.pubkey)) return -EINVAL; ret = hpre_msg_request_set(ctx, req, true); - if (ret) + if (unlikely(ret)) return ret; msg->dw0 |= cpu_to_le32(HPRE_ALG_NC_NCRT); msg->key = cpu_to_le64((u64)ctx->rsa.dma_pubkey); ret = hpre_hw_data_init(hpre_req, req->src, req->src_len, 1, 0); - if (ret) + if (unlikely(ret)) goto clear_all; ret = hpre_hw_data_init(hpre_req, req->dst, req->dst_len, 0, 0); - if (ret) + if (unlikely(ret)) goto clear_all; do { @@ -690,7 +690,7 @@ static int hpre_rsa_enc(struct akcipher_request *req) } while (ret == -EBUSY && ctr++ < HPRE_TRY_SEND_TIMES); /* success */ - if (!ret) + if (likely(!ret)) return -EINPROGRESS; clear_all: @@ -719,11 +719,11 @@ static int hpre_rsa_dec(struct akcipher_request *req) return ret; } - if (!ctx->rsa.prikey) + if (unlikely(!ctx->rsa.prikey)) return -EINVAL; ret = hpre_msg_request_set(ctx, req, true); - if (ret) + if (unlikely(ret)) return ret; if (ctx->crt_g2_mode) { @@ -737,11 +737,11 @@ static int hpre_rsa_dec(struct akcipher_request *req) } ret = hpre_hw_data_init(hpre_req, req->src, req->src_len, 1, 0); - if (ret) + if (unlikely(ret)) goto clear_all; ret = hpre_hw_data_init(hpre_req, req->dst, req->dst_len, 0, 0); - if (ret) + if (unlikely(ret)) goto clear_all; do { @@ -749,7 +749,7 @@ static int hpre_rsa_dec(struct akcipher_request *req) } while (ret == -EBUSY && ctr++ < HPRE_TRY_SEND_TIMES); /* success */ - if (!ret) + if (likely(!ret)) return -EINPROGRESS; clear_all: