From patchwork Mon Jun 15 23:52:49 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victoria Milhoan X-Patchwork-Id: 6612541 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Original-To: patchwork-linux-crypto@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id A3F489F399 for ; Tue, 16 Jun 2015 00:08:41 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 4667A207D5 for ; Tue, 16 Jun 2015 00:08:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CF57D207D1 for ; Tue, 16 Jun 2015 00:08:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751132AbbFPAIi (ORCPT ); Mon, 15 Jun 2015 20:08:38 -0400 Received: from mail-bl2on0131.outbound.protection.outlook.com ([65.55.169.131]:22255 "EHLO na01-bl2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750911AbbFPAIh (ORCPT ); Mon, 15 Jun 2015 20:08:37 -0400 Received: from DM2PR03CA0038.namprd03.prod.outlook.com (10.141.96.37) by BLUPR03MB389.namprd03.prod.outlook.com (10.141.78.11) with Microsoft SMTP Server (TLS) id 15.1.195.6; Mon, 15 Jun 2015 23:53:09 +0000 Received: from BL2FFO11FD048.protection.gbl (2a01:111:f400:7c09::102) by DM2PR03CA0038.outlook.office365.com (2a01:111:e400:2428::37) with Microsoft SMTP Server (TLS) id 15.1.190.14 via Frontend Transport; Mon, 15 Jun 2015 23:53:08 +0000 Authentication-Results: spf=fail (sender IP is 192.88.168.50) smtp.mailfrom=freescale.com; freescale.mail.onmicrosoft.com; dkim=none (message not signed) header.d=none; Received-SPF: Fail (protection.outlook.com: domain of freescale.com does not designate 192.88.168.50 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.168.50; helo=tx30smr01.am.freescale.net; Received: from tx30smr01.am.freescale.net (192.88.168.50) by BL2FFO11FD048.mail.protection.outlook.com (10.173.161.210) with Microsoft SMTP Server (TLS) id 15.1.190.9 via Frontend Transport; Mon, 15 Jun 2015 23:53:08 +0000 Received: from stc-hedley.am.freescale.net (stc-hedley.am.freescale.net [10.67.70.12]) by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id t5FNr7ou003741; Mon, 15 Jun 2015 16:53:08 -0700 From: Victoria Milhoan To: CC: , , Subject: [PATCH RFC 01/11] crypto: caam - Add cache coherency support Date: Mon, 15 Jun 2015 16:52:49 -0700 Message-ID: <1434412379-11623-2-git-send-email-vicki.milhoan@freescale.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1434412379-11623-1-git-send-email-vicki.milhoan@freescale.com> References: <1434412379-11623-1-git-send-email-vicki.milhoan@freescale.com> X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD048; 1:gjbLUTbYZbTY/cOFgNy5UV22F0y19ULDhLDluyDMCs4QNzwU4gclzZwZ/nW189ySlfQtpPeclPxy5DxhZ5BGZ8Ysw3mtmBqAk+zBScBohOythi/1PFv6UOW5AG1vjFHi6P5b+PYrloPQgPhztOqTHHQnlwAO9a188ps2k/UdILwt8H0Pkf10U2bNTZlKsYoYzVibQoaLQgbAykDaN02xItQ5P9Pw6Qq2a1PZQyCzerBehapBfYER9iRt3Kc4zEwesFxOW+at2nUWjs1KdA4diPeVr1xUUAGg2oEHmYZo8oHUyMw1e7xr3iX56N3Evd9KYzoE5/EfsflJ9dFbNlk/xCHjxhnM0dQG2ZDZILL+XFE= X-Forefront-Antispam-Report: CIP:192.88.168.50; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10019020)(6009001)(339900001)(189002)(199003)(62966003)(104016003)(106466001)(47776003)(86362001)(2950100001)(77156002)(229853001)(189998001)(2351001)(105606002)(33646002)(85426001)(76176999)(50986999)(92566002)(50226001)(19580405001)(19580395003)(107886002)(5001960100002)(87936001)(110136002)(48376002)(50466002)(5001920100001)(77096005)(36756003)(6806004)(46102003)(4001430100001)(217873001); DIR:OUT; SFP:1102; SCL:1; SRVR:BLUPR03MB389; H:tx30smr01.am.freescale.net; FPR:; SPF:Fail; MLV:sfv; MX:1; A:1; LANG:en; MIME-Version: 1.0 X-Microsoft-Exchange-Diagnostics: 1; BLUPR03MB389; 2:L8QA9yXjhMmzABtuWZxv8KcEFsPuukmNdIb5pd1KOeAWuHbU6xCjajxfxFfyer5z; 3:VTQDm2wlCewrswlKbm3CZy09FMiiCjrK48PHtFk0+t5eQVfO0v8CDtr913aq1r4f13WQ83gS6ADIH9pXYblae2LfLVmkX/ZydzwPlbIjgM1r/6aEmlTbdZ3/I1FVHhCtxftPZofCQo/cwYaX21OvR4wfa7WxTgCesmYI1ASBPoJCEsX5nXubHW3I69wUTkx8HnsNdTzczZqM/5xw2pCiWh04uZuon68FwzJBaRfluVg=; 20:3lFC76EawYflAPKHdn+xvoBPpwxL2HxTgzpS9bMtT1c1683+iowJJvsS/Golicbn2WDxbmoHFp3opxb3d8UfDZRf6VpGuSUrSBYGOLzEg+kAQ47Skkd54e18Qs+1Brg1ysVewaW81vqATXSx+uIEguo++bNv0JwtKQ7oAdRaPkqF647DEfY6QDhZBDyQZ7sz0cp/ofnLP/VXqObbt1Cyr6aVzYKUX+I3P2NkJL1M1vmfoaGNURX6plCW2Fch3lKj4SN53H4ypCaSQd7+8CT5QkhXsnc2DRiAQHMvqYWnLaZAWi+GT9GKhHIiQxxK6Q+xxbU74IuSRgzBtXKm6Isf4iVTMt4+zBQOZvajZrqUlmQ=; 4:W+sFc22lxtt/8aO+kT3RsefsP2mGE++Eu3flBuJlC8PlWD8G/nkv5rIKkDe3RvsBPVHc2lQbOX648sMaQCBPwjQgxvmQ2c8xT4kuyzCR23SiNHF/JpMuSn0zdplMm13ZQhgJGyhYiAE4MhhpbCLahZTiRbRbv6dsLvWnzlhB/IMPYiJgoxK/r6ho0Tet48ulqaCynSOlUfDxSJi2/vLA4ccgieflX+37/dX7UXhAkSCA6iPPwfFYpvOZHPMjOzFyqCNTtTjLe oAdE6KlydWctcWSyKbkE8PUGxT/dub3AaM= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BLUPR03MB389; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(5005006)(3002001); SRVR:BLUPR03MB389; BCL:0; PCL:0; RULEID:; SRVR:BLUPR03MB389; X-Forefront-PRVS: 0608DEDB67 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BLUPR03MB389; 23:R8J4RCgDpYizJGRAZiRqC9HFtAc7Te3tG/lqUR3nqa?= =?us-ascii?Q?T8jC1/38uxiaRh7ULDygy+d/G2HI+KU0iiR9igGX8tsrVke5R7mVgvy4I8Fi?= =?us-ascii?Q?JG2FrjoEVfOs4bHvOCGQJjoInWYodeFuysaLIm6HI/3YoFS0f8VxEIRT8830?= =?us-ascii?Q?1F0sFun1pOtbcgnt3w4Gq0TPYk68a8hI6A11SOF3jNZSDI5wKhWqkaxrrUTU?= =?us-ascii?Q?X4wlstGSI45Slb9p6A6nG4k8HBVr/MKB9748a0T6KxNwVFY1eZ/5ChGeFmJW?= =?us-ascii?Q?quT52c9JJ0M8aA3MVuLUq9zMF0udvPwnDZP9YBi+OU3GUdV4DEmODH1gPtc9?= =?us-ascii?Q?gZY0wwzwsIOowwuVUNZPkTkCVm2HxXhZFW8YuLj8HWAI7BZK+gv10XVNAyd+?= =?us-ascii?Q?ClFbI6OaEl0FV/HxlAntz8dq6MjE7RHQklHIt1jODjOFifXQbJHCLuYV6Lq/?= =?us-ascii?Q?nSjJyJ11CfS7JxJIfbftDAZHZC6SW8RPQsz8Ib19yXV6WZXfagxAXlvPBmm1?= =?us-ascii?Q?46cyW7wk7VrRgVjDMOZGByLB0w/1hsG6iTAsTavQTfiNEPFyM3/1kxqvcpG5?= =?us-ascii?Q?Dwe5Xy38RBC825zYC6iRDMnFiFoMheRLpM+mRd8duYMXaqKZabFxqzqcvnbp?= =?us-ascii?Q?65f8gEEQcA78/ToGGGU0rmLtXqi3UdKBvt16naoVWPneAQ1JiP33jMLX+5ZF?= =?us-ascii?Q?Ruo7/0ZO21vKzywhiy+bfgcY4LU29pFljJrlkAe0OqiIMHW/XLLdQ0HY0M+M?= =?us-ascii?Q?IopnFzWE+5XPeDyDU+iG3XKed7KN+Dgj76vWnnOmiq28Cek5W1qMvdgIkoHp?= =?us-ascii?Q?36dkCAljqFfPSpc23OM6kpJJcVizVbWio2Ssh6WAEKXqSrXxDv8N+RylXDaZ?= =?us-ascii?Q?9dmvJHAbDda18z67ESadj5uzP76SRGROBA0ZfQfH6wz8oon1etYK2Yds0MKK?= =?us-ascii?Q?QxqX1+zrDweHEnx77bfhltrz0BXtVK6EeDy5z/GQ=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; BLUPR03MB389; 5:MGZgDrGM9Cps76iExqZaIrwbDx7g2YAuqqSUm3kTzBwXzg7lcoM5X5x/5DtY8LMYbV6vQQNtaXg37t6/wNT/KkTGJPdrg2Izmpvfx4XPXsEKCMQ9aS5PdvK27xFfGTvFZjq5YkSaKMiaENcm302hnw==; 24:Wed0o2O5wLu/mgCANBdYNtqBizLaJ07V/9N8mNMfq2eDMakoz+MhuTCdRKvZpHD1pT3KDpoWdkMKr6X+R7WAiuI0mSJTG4ohlqj0PRHtvwg=; 20:flD7p91ZNA4BL5zYuJQv+pddFLDpiTHIsBegBRkMLH8DAgEXFyMFke9yjRbEv0zJ6nctqOmwf78CEtGNGp+o8Q== X-OriginatorOrg: freescale.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jun 2015 23:53:08.7080 (UTC) X-MS-Exchange-CrossTenant-Id: 710a03f5-10f6-4d38-9ff4-a80b81da590d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=710a03f5-10f6-4d38-9ff4-a80b81da590d; Ip=[192.88.168.50]; Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR03MB389 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org X-Spam-Status: No, score=-7.4 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Freescale i.MX6 ARM platforms do not support hardware cache coherency. This patch adds cache coherency support to the CAAM driver. Signed-off-by: Victoria Milhoan --- drivers/crypto/caam/caamhash.c | 28 +++++++++++++++++---------- drivers/crypto/caam/caamrng.c | 10 +++++++++- drivers/crypto/caam/jr.c | 42 +++++++++++++++++++++++++++++++++++++++- drivers/crypto/caam/key_gen.c | 4 +++- drivers/crypto/caam/sg_sw_sec4.h | 1 + 5 files changed, 72 insertions(+), 13 deletions(-) diff --git a/drivers/crypto/caam/caamhash.c b/drivers/crypto/caam/caamhash.c index ba0532e..1662c65 100644 --- a/drivers/crypto/caam/caamhash.c +++ b/drivers/crypto/caam/caamhash.c @@ -500,6 +500,7 @@ static int hash_digest_key(struct caam_hash_ctx *ctx, const u8 *key_in, #endif } dma_unmap_single(jrdev, src_dma, *keylen, DMA_TO_DEVICE); + dma_sync_single_for_cpu(jrdev, dst_dma, digestsize, DMA_FROM_DEVICE); dma_unmap_single(jrdev, dst_dma, digestsize, DMA_FROM_DEVICE); *keylen = digestsize; @@ -608,8 +609,11 @@ static inline void ahash_unmap(struct device *dev, if (edesc->src_nents) dma_unmap_sg_chained(dev, req->src, edesc->src_nents, DMA_TO_DEVICE, edesc->chained); - if (edesc->dst_dma) + if (edesc->dst_dma) { + dma_sync_single_for_cpu(dev, edesc->dst_dma, dst_len, + DMA_FROM_DEVICE); dma_unmap_single(dev, edesc->dst_dma, dst_len, DMA_FROM_DEVICE); + } if (edesc->sec4_sg_bytes) dma_unmap_single(dev, edesc->sec4_sg_dma, @@ -624,8 +628,12 @@ static inline void ahash_unmap_ctx(struct device *dev, struct caam_hash_ctx *ctx = crypto_ahash_ctx(ahash); struct caam_hash_state *state = ahash_request_ctx(req); - if (state->ctx_dma) + if (state->ctx_dma) { + if ((flag == DMA_FROM_DEVICE) || (flag == DMA_BIDIRECTIONAL)) + dma_sync_single_for_cpu(dev, state->ctx_dma, + ctx->ctx_len, flag); dma_unmap_single(dev, state->ctx_dma, ctx->ctx_len, flag); + } ahash_unmap(dev, edesc, req, dst_len); } @@ -807,7 +815,7 @@ static int ahash_update_ctx(struct ahash_request *req) * allocate space for base edesc and hw desc commands, * link tables */ - edesc = kmalloc(sizeof(struct ahash_edesc) + DESC_JOB_IO_LEN + + edesc = kzalloc(sizeof(struct ahash_edesc) + DESC_JOB_IO_LEN + sec4_sg_bytes, GFP_DMA | flags); if (!edesc) { dev_err(jrdev, @@ -918,7 +926,7 @@ static int ahash_final_ctx(struct ahash_request *req) sec4_sg_bytes = (1 + (buflen ? 1 : 0)) * sizeof(struct sec4_sg_entry); /* allocate space for base edesc and hw desc commands, link tables */ - edesc = kmalloc(sizeof(struct ahash_edesc) + DESC_JOB_IO_LEN + + edesc = kzalloc(sizeof(struct ahash_edesc) + DESC_JOB_IO_LEN + sec4_sg_bytes, GFP_DMA | flags); if (!edesc) { dev_err(jrdev, "could not allocate extended descriptor\n"); @@ -1005,7 +1013,7 @@ static int ahash_finup_ctx(struct ahash_request *req) sizeof(struct sec4_sg_entry); /* allocate space for base edesc and hw desc commands, link tables */ - edesc = kmalloc(sizeof(struct ahash_edesc) + DESC_JOB_IO_LEN + + edesc = kzalloc(sizeof(struct ahash_edesc) + DESC_JOB_IO_LEN + sec4_sg_bytes, GFP_DMA | flags); if (!edesc) { dev_err(jrdev, "could not allocate extended descriptor\n"); @@ -1091,7 +1099,7 @@ static int ahash_digest(struct ahash_request *req) sec4_sg_bytes = src_nents * sizeof(struct sec4_sg_entry); /* allocate space for base edesc and hw desc commands, link tables */ - edesc = kmalloc(sizeof(struct ahash_edesc) + sec4_sg_bytes + + edesc = kzalloc(sizeof(struct ahash_edesc) + sec4_sg_bytes + DESC_JOB_IO_LEN, GFP_DMA | flags); if (!edesc) { dev_err(jrdev, "could not allocate extended descriptor\n"); @@ -1165,7 +1173,7 @@ static int ahash_final_no_ctx(struct ahash_request *req) int sh_len; /* allocate space for base edesc and hw desc commands, link tables */ - edesc = kmalloc(sizeof(struct ahash_edesc) + DESC_JOB_IO_LEN, + edesc = kzalloc(sizeof(struct ahash_edesc) + DESC_JOB_IO_LEN, GFP_DMA | flags); if (!edesc) { dev_err(jrdev, "could not allocate extended descriptor\n"); @@ -1245,7 +1253,7 @@ static int ahash_update_no_ctx(struct ahash_request *req) * allocate space for base edesc and hw desc commands, * link tables */ - edesc = kmalloc(sizeof(struct ahash_edesc) + DESC_JOB_IO_LEN + + edesc = kzalloc(sizeof(struct ahash_edesc) + DESC_JOB_IO_LEN + sec4_sg_bytes, GFP_DMA | flags); if (!edesc) { dev_err(jrdev, @@ -1352,7 +1360,7 @@ static int ahash_finup_no_ctx(struct ahash_request *req) sizeof(struct sec4_sg_entry); /* allocate space for base edesc and hw desc commands, link tables */ - edesc = kmalloc(sizeof(struct ahash_edesc) + DESC_JOB_IO_LEN + + edesc = kzalloc(sizeof(struct ahash_edesc) + DESC_JOB_IO_LEN + sec4_sg_bytes, GFP_DMA | flags); if (!edesc) { dev_err(jrdev, "could not allocate extended descriptor\n"); @@ -1447,7 +1455,7 @@ static int ahash_update_first(struct ahash_request *req) * allocate space for base edesc and hw desc commands, * link tables */ - edesc = kmalloc(sizeof(struct ahash_edesc) + DESC_JOB_IO_LEN + + edesc = kzalloc(sizeof(struct ahash_edesc) + DESC_JOB_IO_LEN + sec4_sg_bytes, GFP_DMA | flags); if (!edesc) { dev_err(jrdev, diff --git a/drivers/crypto/caam/caamrng.c b/drivers/crypto/caam/caamrng.c index 26a544b..a8c4af9 100644 --- a/drivers/crypto/caam/caamrng.c +++ b/drivers/crypto/caam/caamrng.c @@ -80,9 +80,12 @@ static struct caam_rng_ctx *rng_ctx; static inline void rng_unmap_buf(struct device *jrdev, struct buf_data *bd) { - if (bd->addr) + if (bd->addr) { + dma_sync_single_for_cpu(jrdev, bd->addr, RN_BUF_SIZE, + DMA_FROM_DEVICE); dma_unmap_single(jrdev, bd->addr, RN_BUF_SIZE, DMA_FROM_DEVICE); + } } static inline void rng_unmap_ctx(struct caam_rng_ctx *ctx) @@ -108,6 +111,10 @@ static void rng_done(struct device *jrdev, u32 *desc, u32 err, void *context) atomic_set(&bd->empty, BUF_NOT_EMPTY); complete(&bd->filled); + + /* Buffer refilled, invalidate cache */ + dma_sync_single_for_cpu(jrdev, bd->addr, RN_BUF_SIZE, DMA_FROM_DEVICE); + #ifdef DEBUG print_hex_dump(KERN_ERR, "rng refreshed buf@: ", DUMP_PREFIX_ADDRESS, 16, 4, bd->buf, RN_BUF_SIZE, 1); @@ -211,6 +218,7 @@ static inline int rng_create_sh_desc(struct caam_rng_ctx *ctx) print_hex_dump(KERN_ERR, "rng shdesc@: ", DUMP_PREFIX_ADDRESS, 16, 4, desc, desc_bytes(desc), 1); #endif + return 0; } diff --git a/drivers/crypto/caam/jr.c b/drivers/crypto/caam/jr.c index b8b5d47..a693bf7 100644 --- a/drivers/crypto/caam/jr.c +++ b/drivers/crypto/caam/jr.c @@ -168,6 +168,9 @@ static void caam_jr_dequeue(unsigned long devarg) void (*usercall)(struct device *dev, u32 *desc, u32 status, void *arg); u32 *userdesc, userstatus; void *userarg; + dma_addr_t outbusaddr; + + outbusaddr = rd_reg64(&jrp->rregs->outring_base); while (rd_reg32(&jrp->rregs->outring_used)) { @@ -177,10 +180,19 @@ static void caam_jr_dequeue(unsigned long devarg) sw_idx = tail = jrp->tail; hw_idx = jrp->out_ring_read_index; + dma_sync_single_for_cpu(dev, outbusaddr, + sizeof(struct jr_outentry) * JOBR_DEPTH, + DMA_FROM_DEVICE); for (i = 0; CIRC_CNT(head, tail + i, JOBR_DEPTH) >= 1; i++) { sw_idx = (tail + i) & (JOBR_DEPTH - 1); + /* + * Ensure that tail is read before using it as part of + * the index into the software ring. + */ + smp_read_barrier_depends(); + if (jrp->outring[hw_idx].desc == jrp->entinfo[sw_idx].desc_addr_dma) break; /* found */ @@ -202,6 +214,13 @@ static void caam_jr_dequeue(unsigned long devarg) userdesc = jrp->entinfo[sw_idx].desc_addr_virt; userstatus = jrp->outring[hw_idx].jrstatus; + /* + * Make sure all information from the job has been obtained + * before telling CAAM that the job has been removed from the + * output ring. + */ + smp_mb(); + /* set done */ wr_reg32(&jrp->rregs->outring_rmvd, 1); @@ -216,6 +235,12 @@ static void caam_jr_dequeue(unsigned long devarg) if (sw_idx == tail) { do { tail = (tail + 1) & (JOBR_DEPTH - 1); + + /* + * Ensure that tail is read before using it to + * update the software ring's tail index + */ + smp_read_barrier_depends(); } while (CIRC_CNT(head, tail, JOBR_DEPTH) >= 1 && jrp->entinfo[tail].desc_addr_dma == 0); @@ -321,7 +346,7 @@ int caam_jr_enqueue(struct device *dev, u32 *desc, struct caam_drv_private_jr *jrp = dev_get_drvdata(dev); struct caam_jrentry_info *head_entry; int head, tail, desc_size; - dma_addr_t desc_dma; + dma_addr_t desc_dma, inpbusaddr; desc_size = (*desc & HDR_JD_LENGTH_MASK) * sizeof(u32); desc_dma = dma_map_single(dev, desc, desc_size, DMA_TO_DEVICE); @@ -330,6 +355,11 @@ int caam_jr_enqueue(struct device *dev, u32 *desc, return -EIO; } + inpbusaddr = rd_reg64(&jrp->rregs->inpring_base); + dma_sync_single_for_device(dev, inpbusaddr, + sizeof(dma_addr_t) * JOBR_DEPTH, + DMA_TO_DEVICE); + spin_lock_bh(&jrp->inplock); head = jrp->head; @@ -351,12 +381,22 @@ int caam_jr_enqueue(struct device *dev, u32 *desc, jrp->inpring[jrp->inp_ring_write_index] = desc_dma; + dma_sync_single_for_device(dev, inpbusaddr, + sizeof(dma_addr_t) * JOBR_DEPTH, + DMA_TO_DEVICE); + smp_wmb(); jrp->inp_ring_write_index = (jrp->inp_ring_write_index + 1) & (JOBR_DEPTH - 1); jrp->head = (head + 1) & (JOBR_DEPTH - 1); + /* + * Ensure that all job information has been written before + * notifying CAAM that a new job was added to the input ring. + */ + wmb(); + wr_reg32(&jrp->rregs->inpring_jobadd, 1); spin_unlock_bh(&jrp->inplock); diff --git a/drivers/crypto/caam/key_gen.c b/drivers/crypto/caam/key_gen.c index e1eaf4f..6481f71 100644 --- a/drivers/crypto/caam/key_gen.c +++ b/drivers/crypto/caam/key_gen.c @@ -71,6 +71,7 @@ int gen_split_key(struct device *jrdev, u8 *key_out, int split_key_len, } init_job_desc(desc, 0); + append_key(desc, dma_addr_in, keylen, CLASS_2 | KEY_DEST_CLASS_REG); /* Sets MDHA up into an HMAC-INIT */ @@ -111,7 +112,8 @@ int gen_split_key(struct device *jrdev, u8 *key_out, int split_key_len, split_key_pad_len, 1); #endif } - + dma_sync_single_for_cpu(jrdev, dma_addr_out, split_key_pad_len, + DMA_FROM_DEVICE); dma_unmap_single(jrdev, dma_addr_out, split_key_pad_len, DMA_FROM_DEVICE); out_unmap_in: diff --git a/drivers/crypto/caam/sg_sw_sec4.h b/drivers/crypto/caam/sg_sw_sec4.h index 3b91821..6365585 100644 --- a/drivers/crypto/caam/sg_sw_sec4.h +++ b/drivers/crypto/caam/sg_sw_sec4.h @@ -98,6 +98,7 @@ static int dma_map_sg_chained(struct device *dev, struct scatterlist *sg, } else { dma_map_sg(dev, sg, nents, dir); } + return nents; }