From patchwork Tue Nov 22 13:44:06 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Horia Geanta X-Patchwork-Id: 9441125 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B4E6A600BA for ; Tue, 22 Nov 2016 13:46:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A78942852E for ; Tue, 22 Nov 2016 13:46:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9C31228533; Tue, 22 Nov 2016 13:46:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AED762852E for ; Tue, 22 Nov 2016 13:46:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755383AbcKVNqL (ORCPT ); Tue, 22 Nov 2016 08:46:11 -0500 Received: from mail-by2nam01on0050.outbound.protection.outlook.com ([104.47.34.50]:4864 "EHLO NAM01-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754804AbcKVNqH (ORCPT ); Tue, 22 Nov 2016 08:46:07 -0500 Received: from DM5PR03CA0020.namprd03.prod.outlook.com (10.175.104.30) by BN1PR0301MB0738.namprd03.prod.outlook.com (10.160.78.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.747.10; Tue, 22 Nov 2016 13:45:06 +0000 Received: from BL2FFO11OLC003.protection.gbl (2a01:111:f400:7c09::103) by DM5PR03CA0020.outlook.office365.com (2603:10b6:3:118::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.734.8 via Frontend Transport; Tue, 22 Nov 2016 13:45:06 +0000 Authentication-Results: spf=fail (sender IP is 192.88.158.2) smtp.mailfrom=nxp.com; nxp.com; dkim=none (message not signed) header.d=none; nxp.com; dmarc=fail action=none header.from=nxp.com; nxp.com; dkim=none (message not signed) header.d=none; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.158.2 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.158.2; helo=az84smr01.freescale.net; Received: from az84smr01.freescale.net (192.88.158.2) by BL2FFO11OLC003.mail.protection.outlook.com (10.173.161.187) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.1.734.4 via Frontend Transport; Tue, 22 Nov 2016 13:45:05 +0000 X-IncomingTopHeaderMarker: OriginalChecksum:; UpperCasedChecksum:; SizeAsReceived:967; Count:13 Received: from enigma.ea.freescale.net (enigma.ea.freescale.net [10.171.77.120]) by az84smr01.freescale.net (8.14.3/8.14.0) with ESMTP id uAMDioJe031527; Tue, 22 Nov 2016 06:45:03 -0700 From: =?UTF-8?q?Horia=20Geant=C4=83?= To: Herbert Xu CC: "David S. Miller" , , Dan Douglass , Tudor Ambarus , Alexandru Porosanu Subject: [PATCH 04/10] crypto: caam - improve key inlining Date: Tue, 22 Nov 2016 15:44:06 +0200 Message-ID: <1479822252-23833-5-git-send-email-horia.geanta@nxp.com> X-Mailer: git-send-email 2.4.4 In-Reply-To: <1479822252-23833-1-git-send-email-horia.geanta@nxp.com> References: <1479822252-23833-1-git-send-email-horia.geanta@nxp.com> MIME-Version: 1.0 X-IncomingHeaderCount: 13 X-EOPAttributedMessage: 0 X-Matching-Connectors: 131242959058554818; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.158.2; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(7916002)(2980300002)(1110001)(1109001)(339900001)(336004)(189002)(199003)(36756003)(23676002)(8676002)(47776003)(33646002)(305945005)(106466001)(38730400001)(50986999)(87936001)(4326007)(92566002)(5820100001)(85426001)(2906002)(76176999)(2870700001)(77096005)(69596002)(6916009)(2950100002)(86362001)(626004)(110136003)(5660300001)(575784001)(104016004)(356003)(7846002)(50466002)(105606002)(8666005)(97736004)(50226002)(81166006)(81156014)(8936002)(189998001)(68736007)(7059030); DIR:OUT; SFP:1101; SCL:1; SRVR:BN1PR0301MB0738; H:az84smr01.freescale.net; FPR:; SPF:Fail; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11OLC003; 1:JbwIe1ETqiGmYdIe3qaoY2zmvykdzOsM4ejacNMqxsmmSJK+ezlAI8JE0aBOuN7UKI8eKn9OGG3FhdsVzARzWX/l/SI19YneGTV7EvZlV63/veKVa180ZGxtA3a1kM21sK7HvupLbrAKw+evWRe0jlkdixgutnShMwTK16Hf1KyfJpdL0c1F7G9zdz1ov9y4SFOQPHkRLO4y7A2TIzwsZ9hMg7Ev6lEXLdw1fF3mPPOjNpiito98vVk3OJNoQKWQKQkt/J7qWSw6iMFmdwjZPjU/PxYmIImg7tRVakJldcRTOTJZOxQdMpQUl2TGd05JaXAarAos1ts5aINYm3OHar2kD3fMFlu/3lGVi5dG0DYzxwoXS1ekD37poAysS302Zstlo5S9H591OhfAUpBIX0ptuFgYpGbEUhNEPeUTh00I4Ps11+hIWu2SFK++lwIIaYHsSx1FpyVS1a5VjyB8EkmfY7OO1bjswd4EaWeds3OhLfkIatqDvKx+Gy8SPjCX4bDU8j6YXOkxnFIO/U2Q7bMw/uMF1oiW/0jA8k/Xnw1+3EsslMHeEppDU8qvgmzNF75ehj1M8MULJsR4mIaSfYHneNXIPv0h4kTu2/5ZYsWMJi0d5dnfRlrU2WyHDD//xM7u4+DfN09QffqA4Lhlxr48iXY6EAYAQFfOVDe9EvXUV4+r0ppMLF7u2zGujdJwVYXmjNBT6ScTt2F1tE5jxkUr5zsxWR1cLGpa3fnvZrI= X-MS-Office365-Filtering-Correlation-Id: adeb48f2-470c-4c97-6e64-08d412ddc469 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:BN1PR0301MB0738; X-Microsoft-Exchange-Diagnostics: 1; BN1PR0301MB0738; 3:j5MXdYIb+leKhhMPJsyk/KlzgpxE2TVVP7U6vunQlXxw8iDQM5frnUEXtoY72Bx0RdQUeMCnXdUTVuvDU9vIqCHOSLAvDJosmveCy9M/r3Cagsn6rmr7ofEHIe/HeIoo1mL3i/dteN6K8dVvGJyN6mRT10sFecDIiyNN0d3sWJK/ZesNnvN/5LiQ24P7HV0xdT105NiPn3ZTMMRFX0UA4uCwzOy+Gq/1/8EQLR+yMOz9bAv1XdHdIdF1pah2USUp7C66F7BrFOLNXoQwwfXt0V77a1Kufw9QcLdlWDbZY6LR0RJCqgUFpXTmh1gH/7pz5uD+2I6icx+UD+IdV6MMpN/0SKohF7EdFzF4nYhXOX96UIOfzt+36h6l0V5ucfus X-Microsoft-Exchange-Diagnostics: 1; BN1PR0301MB0738; 25:SBArvOkU3tSDT9HeX2ZyPj/lNqcKCFBSTxI3XPVFPj2y9Ie75v1opgel0gPGkcuGkrmp9XbDhGOFMZghT5zWlY7oBBbnp1GtAifYAx+cL5H5wEbROCXNidQVoVmszWzIW+SwzSFHwy/hNbCPrUPBj21rYqq6ZggvGT4aRwphG3vlQjkb14H1Do7PN/1w/bB2Hx8yRxnlwN30xGCPX11gWxmH4Yr1xWmFvIg/yCx+qPsNCUm3J9258q7BgBJRzjFiM+c15gtkMnjQAehNhSHJgB77NW52C/BZBmNWhQVt1V0SoHfyHkdg/e1YbFG0E0QGX4jTt6UD7nq/QtkJXe2h4AmQFmqddX+yO7SkDxl2FJA7nxm/Cjid45Ve85w6AJc/iY3b6BJjMlHfakhb7PcYXcjjhu/OaC5NhY3BA/Xb0Ai9R4KCm1cL3xRRpe699TsU5pJ3i4bpPlqs2vzzvctOdkSmnwHdjxLDYA2MQNDnGSWUTfIABnc6gWC3Jm9DOAICHimufXmZ5vRwboMsDEvAw1ihxLWX6HKZ6LGJpRkKyYeUH3JKmS6fu8KfHfADUAHxkylAfmUjuyELT+1q5LYASPi27SFyu6GUIQVgGaMss5xZez3qtHvs8ZkofVb4hl5orPRvJme/857GhZJaoU+5RxtSli/ZEUsIZNk4jP4NzfTrIQwZ1pOdI+Vp+b+6yU/Ha8x73O9vxlDNE2yauT1zKZYnqX2sEzxfsvzNXeF7eFBoSZEnklol9of+PcBF+LDfN/Xh3ZUyza7s/6ZJpQNNZw== X-Microsoft-Exchange-Diagnostics: 1; BN1PR0301MB0738; 31:pIEGBr1ZafBFuZye3SVIlc4ktEQmtT5rgVKYgNtEbmgRYHN738YtmVK3uhg3RF/wgSn1b/kMRS80nWNG86XRwEsPSB8oN9WHG14WFnjJXGWoKl1x9iYdfAU4URoPMIPLdtefSnHx4+e/T+bSwYAHwtb8oiqTSvjf3uLQWy93mHxeh8H6oSRTHcCTsyeRv+3Ej6VLBGS6/fOzQq7TyeO06NZVi/3puKl4L7+lYgj+mj3PQhGSfRb+wkHwRKsOQCh6wBvj99cJ7L6J3Ub8W31uL7s8sOf6NhZDEdnv/oD89YM=; 4:tD4wvI5DbSf1WkAWByjzLmCX79uGtgc+dWGMJxa8kfTJM1GYbZmlvEKbYb29O1fA0+TPzwCQySypmw67efxIXchfUvwMTF0GWbgn1+RvoywOqZw8fnimbOfPyb0PPgOdNCcj0UXMZBRF32iVes6ssEbuolYco2I1sJhxkIpVdlsOrzZ41MuJqmf5PIyGKLDhNr1EKpFMQnXzBtyfbLPwb0HgPfcS/1zHpk3fI9xKZeJQgfLwp5gprSySEAoX5wRf+qTsxJaUcyocVjruDcNxNHtcFrPsx12MVyNGwjtxo7rlh0ngpo2RUpP5Bm4pAboOTE3ZruF6RBqZ++7ml6pnXxDeCFNIalWxOKZaGg/pcBbrU+gN4EfWjoNuqfWweUpbDG6N+5jnFlLRAVuw1WBi67KYY26NzJWJCHmqI0XyJAp6f+1yAuMhJBL/ej3p0NuQuYxtF07fufbG26w369gAvJRLqc4Mijeoxn0W5W5aYe0DXMGcDRIMSiaFc1WAkXMrQBBK3qAi/HtjXYKABz4YWkRJrj1k9FQfZ6PKx6Corc5deTzeWwwVjey13KBJ1rtV X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6095035)(601004)(2401047)(8121501046)(5005006)(13023025)(13024025)(13017025)(13018025)(13015025)(3002001)(10201501046)(6055026)(6096035); SRVR:BN1PR0301MB0738; BCL:0; PCL:0; RULEID:(400006); SRVR:BN1PR0301MB0738; X-Forefront-PRVS: 0134AD334F X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTjFQUjAzMDFNQjA3Mzg7MjM6VElXaG95cDhMcFNaYWtyQTFlclg3aFNk?= =?utf-8?B?amVYOVNuUmVaNEhqckZHd3htcDFwNkQ1bVRUNXQvU0s2SmN0c1RuMElQck5m?= =?utf-8?B?NllCbmIzVmJNZVB6K0Q0aHF2Zm0vWHp6RTl0ak5Ta3NUUVg3L0hqVG1zZzZq?= =?utf-8?B?Mm9wbnFsYk50ZU5kQ2doZ1UzSEdYSTBGZ2ZwQlBKeDBiT0NFR3FsY0NKeEt0?= =?utf-8?B?WE9yL2NmU1hZY1ZnQWhoQm42djRnYnVRNXlBaXV2WHpJRFBpSXdGMlJhSklQ?= =?utf-8?B?V0t1bzRaSkhBQlJMSE9lTmdGWkVtNUpNN2xpVEE2dlZmZHdtcHZvY3duYVZD?= =?utf-8?B?L3pSRHFzOUpCQjVxeGlYWE1BTVFlNVk1M1pxT2dBeG9pc2g5cVFhdXE5ZEd3?= =?utf-8?B?YjMvRzZ2V3RVQkJVWEYrRFR5cWVUS3U5TFBHSTdUb2FTc2FGS3Jwc3MyTWhT?= =?utf-8?B?ZDAxSTFyOW9YSTUrVjhpMm80OGVScGZwRFJvbU9mVzVCellHY20wQjEvV0Y5?= =?utf-8?B?Vk1NVUF1M1RCTFFhOHVnWU0yejh5VWFEakw5a0lNZ0Nrd2V2TGdkdzlsRkJS?= =?utf-8?B?QWd0ZE5Janh2WU03a2R1bnFERnZqWVVROVFPcytVKzg1eTJJdzExMXpONkFG?= =?utf-8?B?RFRYNVpYdjM1UklCQ0w0Q09USXVoQnBDMXd0MzE5TzRHSWlMMWMyUU9yNlhp?= =?utf-8?B?emRKa1ZQaUFrQ0pRWE42TjhwWWxMY0J2ZFA0Q1U1NWM2cmNKTDhEUUdBdXJv?= =?utf-8?B?b0E0NE5ESkVxOGZIVGsveVlTQTVEQnYrZUw5UGJQYmdkQ0xKTFI2VDRRZ2lD?= =?utf-8?B?RVBheTB1Uk9qOFc0Ylo3SGxqZjJjaU51Z0RVYTlyT2hXbkRmSURDek03K3Fp?= =?utf-8?B?cHNCT2hJMXh3YU45UGpsS3ZTWldBd25hNGpKU2dwbFhJTFErKzV3Y291cU1T?= =?utf-8?B?dkdFYmUvbW9YY0JlYWZ5VTZLdmZ5Mnd1REl1a04zWFZRTzY1SWdZcjBIY0NV?= =?utf-8?B?YnhzNERLT0RqOWx2KytkNFF0VFpvdjlkdGU1b0N1NjQrYkg0SFdvWXNwcmZr?= =?utf-8?B?V0lsMHBqVTFGZ1NxdFdtbjVUSVBXK2dEVFNVWi81ZkFZVjFGRG9ubllBNFBM?= =?utf-8?B?SUNqbkZ1OEZPd0ZIbnBzeXA2eGpYTTVtTmlMU05kUGFZTDZaZDZNUkk3cXZE?= =?utf-8?B?UVhNRTVrTUpkV0V4dGdVWUlOdzJVK3lLRzZCS0F6c3pVTnBBV3dEaHJGZS9D?= =?utf-8?B?aEhqQU50MHJlR2pkUjQzbHRZRE45Z2tRWlh0K0tUTi9KNEhReG53cmRTa0xo?= =?utf-8?B?TWVzdjFrKzdHNDkxYUI3V24yL1hSeGN0cmlCZUtNaDBQT3RWeTlGTEJXbTlX?= =?utf-8?B?VTNvdU1CVGZIbFozTTJyUmg5dWNMa0oxSXo4em14R2tPUjVnSkp5MlhaWnFo?= =?utf-8?B?aDI4eTBia01wODZtb3Q2YUNqOGFXcEQyNGt0UWkxUHg5b0w4RUlwQjNkY05I?= =?utf-8?B?UkZtMlorcDBtT1NOQWY1ZEJyb3hENXdVc1Y4TXgwY1ptZ3JYWkZsTFEycnRU?= =?utf-8?B?aVhpeDdUbCtzOXppQ2s1Mk9CcFFIVWZPZz09?= X-Microsoft-Exchange-Diagnostics: 1; BN1PR0301MB0738; 6:C8hGuw+kAizKv0dKkBnqjbulohasdmYJ4pXIk8Jj5fpkXhvQqWKYkrBEGKjImiHXV3DxWgvfVbt42pfal8p9lnU/tJ8YE9q8BpzFCuRx4Ij+J1zK0oVkTl4BeifppX9Xg/BSDorTykgDPhJ5orOgvTBcn9pN6jhJWsbTXkpcMz46DHrKuXcW2+PWPV+3dCXZO5/7gmH344Yo8JjwYLL+pC+PGGsWFLT6LzMJig3pnTOfid1p5j/4XMS39oKRovEpUDDy5mNWTGgLyt4KZ3qnYWS8U9G2kR1XrnQ0otwKGmUiUnw9cU3aNPjxE8fcA2thQ2Wk07u/bp9xheISH1XUpv6wKlhYOeCV4SJa3IGX7/VtgsLNWE4FHopoxr0OTHWEosMJkLpKiyhpT9Q1ltZMFoWPXv+zIYTOwr7xe6LBh6LdxwyfWySuxwmFUyMdT9Ju; 5:rlBt0hbYLMWQUFwauXD5bf5od/MzaGOvgZAl5GtDD6R68AV0HL9EN9WD1t1jSD+KDxrUC65vEfNTTXZ79/Eny8zhZ0AqHduhMREqWDhmSHWni6Yy/R+lcRF4u1Svj1tvFaw+goSqK/IdszwuLJ/dp2G2rbwe9IIpbvHIHSOl/Siq18hMhsD06/B5D99jmvXq; 24:sgSoypivDoZXay67mLjWfUeKwrGh7qAH7Uh88Kdnkk79MnJyoFYyKjlrjIrG9+jfiN0iq/+0ZgsQW88yo4UeYBiJJ/OMVN8FGFCbXHGH/Sc= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BN1PR0301MB0738; 7:dvNoooJjdKdrC6gw02zw84jujH9PkCCeKrda7lWJt7T0bfwTS24EzKEO2CqWZD2VdbuDuItxScbHZwCmjmR7PFKWwLX+mNLDjBiX+OU2YgRYyO+UGqAu1V967EBat08/xYhub9QUyU1puQswjMbDuYEUbJjBn/zn4LdncZZJEvuV747k6yrNKUCCtnDpq+rba3xUrCTCQKbrJLWl1UGHkye3u7h1oykDe3ZTkF7F6MFmemXGUVEMm/c/FZsvlUBzXHxU3Q17gO0oXoUPIrPCgjoIi2QdfbneMB8QcRqaxLb3A8iTd+SZDGsDUgGt304zRS07m8z2WpyA2kWLOMoG+gqFNZT/cLfB75cTNnDyuoLweyrrTsz8ymMI8y2F2E6O9yuzGdqOrql+ijzEW+EWwniPJUvujbD78+rUw2E4TJrvH7ROjRZ78T6pTc/D2Qv9c/2B9CaOLdUUA9BVKZxNXQ== X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Nov 2016 13:45:05.6214 (UTC) X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.158.2]; Helo=[az84smr01.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN1PR0301MB0738 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For authenc / stitched AEAD algorithms, check independently each of the two (authentication, encryption) keys whether inlining is possible. Prioritize the inlining of the authentication key, since the length of the (split) key is bigger than that of the encryption key. For the other algorithms, compute only once per tfm the remaining available bytes and decide whether key inlining is possible based on this. Signed-off-by: Horia Geantă --- drivers/crypto/caam/caamalg.c | 130 ++++++++++++++++++++++---------------- drivers/crypto/caam/desc_constr.h | 39 ++++++++++++ 2 files changed, 116 insertions(+), 53 deletions(-) diff --git a/drivers/crypto/caam/caamalg.c b/drivers/crypto/caam/caamalg.c index 48fc000d86bf..5f332df1a8e6 100644 --- a/drivers/crypto/caam/caamalg.c +++ b/drivers/crypto/caam/caamalg.c @@ -224,7 +224,7 @@ struct caam_ctx { }; static void init_sh_desc_key_aead(u32 *desc, struct caam_ctx *ctx, - int keys_fit_inline, bool is_rfc3686) + bool is_rfc3686) { u32 *key_jump_cmd; unsigned int enckeylen = ctx->cdata.keylen; @@ -244,18 +244,20 @@ static void init_sh_desc_key_aead(u32 *desc, struct caam_ctx *ctx, if (is_rfc3686) enckeylen -= CTR_RFC3686_NONCE_SIZE; - if (keys_fit_inline) { + if (ctx->adata.key_inline) append_key_as_imm(desc, (void *)ctx->adata.key, ctx->adata.keylen_pad, ctx->adata.keylen, CLASS_2 | KEY_DEST_MDHA_SPLIT | KEY_ENC); - append_key_as_imm(desc, (void *)ctx->cdata.key, enckeylen, - enckeylen, CLASS_1 | KEY_DEST_CLASS_REG); - } else { + else append_key(desc, ctx->adata.key, ctx->adata.keylen, CLASS_2 | KEY_DEST_MDHA_SPLIT | KEY_ENC); + + if (ctx->cdata.key_inline) + append_key_as_imm(desc, (void *)ctx->cdata.key, enckeylen, + enckeylen, CLASS_1 | KEY_DEST_CLASS_REG); + else append_key(desc, ctx->cdata.key, enckeylen, CLASS_1 | KEY_DEST_CLASS_REG); - } /* Load Counter into CONTEXT1 reg */ if (is_rfc3686) { @@ -282,13 +284,14 @@ static int aead_null_set_sh_desc(struct crypto_aead *aead) struct device *jrdev = ctx->jrdev; u32 *key_jump_cmd, *jump_cmd, *read_move_cmd, *write_move_cmd; u32 *desc; + int rem_bytes = CAAM_DESC_BYTES_MAX - AEAD_DESC_JOB_IO_LEN - + ctx->adata.keylen_pad; /* * Job Descriptor and Shared Descriptors * must all fit into the 64-word Descriptor h/w Buffer */ - if (DESC_AEAD_NULL_ENC_LEN + AEAD_DESC_JOB_IO_LEN + - ctx->adata.keylen_pad <= CAAM_DESC_BYTES_MAX) { + if (rem_bytes >= DESC_AEAD_NULL_ENC_LEN) { ctx->adata.key_inline = true; ctx->adata.key = (uintptr_t)ctx->key; } else { @@ -368,8 +371,7 @@ static int aead_null_set_sh_desc(struct crypto_aead *aead) * Job Descriptor and Shared Descriptors * must all fit into the 64-word Descriptor h/w Buffer */ - if (DESC_AEAD_NULL_DEC_LEN + DESC_JOB_IO_LEN + - ctx->adata.keylen_pad <= CAAM_DESC_BYTES_MAX) { + if (rem_bytes >= DESC_AEAD_NULL_DEC_LEN) { ctx->adata.key_inline = true; ctx->adata.key = (uintptr_t)ctx->key; } else { @@ -463,10 +465,11 @@ static int aead_set_sh_desc(struct crypto_aead *aead) unsigned int ivsize = crypto_aead_ivsize(aead); struct caam_ctx *ctx = crypto_aead_ctx(aead); struct device *jrdev = ctx->jrdev; - bool keys_fit_inline; u32 geniv, moveiv; u32 ctx1_iv_off = 0; u32 *desc; + u32 inl_mask; + unsigned int data_len[2]; const bool ctr_mode = ((ctx->cdata.algtype & OP_ALG_AAI_MASK) == OP_ALG_AAI_CTR_MOD128); const bool is_rfc3686 = alg->caam.rfc3686; @@ -493,6 +496,9 @@ static int aead_set_sh_desc(struct crypto_aead *aead) if (is_rfc3686) ctx1_iv_off = 16 + CTR_RFC3686_NONCE_SIZE; + data_len[0] = ctx->adata.keylen_pad; + data_len[1] = ctx->cdata.keylen; + if (alg->caam.geniv) goto skip_enc; @@ -500,24 +506,30 @@ static int aead_set_sh_desc(struct crypto_aead *aead) * Job Descriptor and Shared Descriptors * must all fit into the 64-word Descriptor h/w Buffer */ - if (DESC_AEAD_ENC_LEN + AUTHENC_DESC_JOB_IO_LEN + - ctx->adata.keylen_pad + ctx->cdata.keylen + - (is_rfc3686 ? DESC_AEAD_CTR_RFC3686_LEN : 0) <= - CAAM_DESC_BYTES_MAX) { - keys_fit_inline = true; + if (desc_inline_query(DESC_AEAD_ENC_LEN + + (is_rfc3686 ? DESC_AEAD_CTR_RFC3686_LEN : 0), + AUTHENC_DESC_JOB_IO_LEN, data_len, &inl_mask, + ARRAY_SIZE(data_len)) < 0) + return -EINVAL; + + if (inl_mask & 1) ctx->adata.key = (uintptr_t)ctx->key; - ctx->cdata.key = (uintptr_t)(ctx->key + ctx->adata.keylen_pad); - } else { - keys_fit_inline = false; + else ctx->adata.key = ctx->key_dma; + + if (inl_mask & 2) + ctx->cdata.key = (uintptr_t)(ctx->key + ctx->adata.keylen_pad); + else ctx->cdata.key = ctx->key_dma + ctx->adata.keylen_pad; - } + + ctx->adata.key_inline = !!(inl_mask & 1); + ctx->cdata.key_inline = !!(inl_mask & 2); /* aead_encrypt shared descriptor */ desc = ctx->sh_desc_enc; /* Note: Context registers are saved. */ - init_sh_desc_key_aead(desc, ctx, keys_fit_inline, is_rfc3686); + init_sh_desc_key_aead(desc, ctx, is_rfc3686); /* Class 2 operation */ append_operation(desc, ctx->adata.algtype | OP_ALG_AS_INITFINAL | @@ -572,24 +584,30 @@ static int aead_set_sh_desc(struct crypto_aead *aead) * Job Descriptor and Shared Descriptors * must all fit into the 64-word Descriptor h/w Buffer */ - if (DESC_AEAD_DEC_LEN + AUTHENC_DESC_JOB_IO_LEN + - ctx->adata.keylen_pad + ctx->cdata.keylen + - (is_rfc3686 ? DESC_AEAD_CTR_RFC3686_LEN : 0) <= - CAAM_DESC_BYTES_MAX) { - keys_fit_inline = true; + if (desc_inline_query(DESC_AEAD_DEC_LEN + + (is_rfc3686 ? DESC_AEAD_CTR_RFC3686_LEN : 0), + AUTHENC_DESC_JOB_IO_LEN, data_len, &inl_mask, + ARRAY_SIZE(data_len)) < 0) + return -EINVAL; + + if (inl_mask & 1) ctx->adata.key = (uintptr_t)ctx->key; - ctx->cdata.key = (uintptr_t)(ctx->key + ctx->adata.keylen_pad); - } else { - keys_fit_inline = false; + else ctx->adata.key = ctx->key_dma; + + if (inl_mask & 2) + ctx->cdata.key = (uintptr_t)(ctx->key + ctx->adata.keylen_pad); + else ctx->cdata.key = ctx->key_dma + ctx->adata.keylen_pad; - } + + ctx->adata.key_inline = !!(inl_mask & 1); + ctx->cdata.key_inline = !!(inl_mask & 2); /* aead_decrypt shared descriptor */ desc = ctx->sh_desc_dec; /* Note: Context registers are saved. */ - init_sh_desc_key_aead(desc, ctx, keys_fit_inline, is_rfc3686); + init_sh_desc_key_aead(desc, ctx, is_rfc3686); /* Class 2 operation */ append_operation(desc, ctx->adata.algtype | OP_ALG_AS_INITFINAL | @@ -660,24 +678,30 @@ static int aead_set_sh_desc(struct crypto_aead *aead) * Job Descriptor and Shared Descriptors * must all fit into the 64-word Descriptor h/w Buffer */ - if (DESC_AEAD_GIVENC_LEN + AUTHENC_DESC_JOB_IO_LEN + - ctx->adata.keylen_pad + ctx->cdata.keylen + - (is_rfc3686 ? DESC_AEAD_CTR_RFC3686_LEN : 0) <= - CAAM_DESC_BYTES_MAX) { - keys_fit_inline = true; + if (desc_inline_query(DESC_AEAD_GIVENC_LEN + + (is_rfc3686 ? DESC_AEAD_CTR_RFC3686_LEN : 0), + AUTHENC_DESC_JOB_IO_LEN, data_len, &inl_mask, + ARRAY_SIZE(data_len)) < 0) + return -EINVAL; + + if (inl_mask & 1) ctx->adata.key = (uintptr_t)ctx->key; - ctx->cdata.key = (uintptr_t)(ctx->key + ctx->adata.keylen_pad); - } else { - keys_fit_inline = false; + else ctx->adata.key = ctx->key_dma; + + if (inl_mask & 2) + ctx->cdata.key = (uintptr_t)(ctx->key + ctx->adata.keylen_pad); + else ctx->cdata.key = ctx->key_dma + ctx->adata.keylen_pad; - } + + ctx->adata.key_inline = !!(inl_mask & 1); + ctx->cdata.key_inline = !!(inl_mask & 2); /* aead_givencrypt shared descriptor */ desc = ctx->sh_desc_enc; /* Note: Context registers are saved. */ - init_sh_desc_key_aead(desc, ctx, keys_fit_inline, is_rfc3686); + init_sh_desc_key_aead(desc, ctx, is_rfc3686); if (is_rfc3686) goto copy_iv; @@ -787,6 +811,8 @@ static int gcm_set_sh_desc(struct crypto_aead *aead) u32 *key_jump_cmd, *zero_payload_jump_cmd, *zero_assoc_jump_cmd1, *zero_assoc_jump_cmd2; u32 *desc; + int rem_bytes = CAAM_DESC_BYTES_MAX - GCM_DESC_JOB_IO_LEN - + ctx->cdata.keylen; if (!ctx->cdata.keylen || !ctx->authsize) return 0; @@ -796,8 +822,7 @@ static int gcm_set_sh_desc(struct crypto_aead *aead) * Job Descriptor and Shared Descriptor * must fit into the 64-word Descriptor h/w Buffer */ - if (DESC_GCM_ENC_LEN + GCM_DESC_JOB_IO_LEN + - ctx->cdata.keylen <= CAAM_DESC_BYTES_MAX) { + if (rem_bytes >= DESC_GCM_ENC_LEN) { ctx->cdata.key_inline = true; ctx->cdata.key = (uintptr_t)ctx->key; } else { @@ -895,8 +920,7 @@ static int gcm_set_sh_desc(struct crypto_aead *aead) * Job Descriptor and Shared Descriptors * must all fit into the 64-word Descriptor h/w Buffer */ - if (DESC_GCM_DEC_LEN + GCM_DESC_JOB_IO_LEN + - ctx->cdata.keylen <= CAAM_DESC_BYTES_MAX) { + if (rem_bytes >= DESC_GCM_DEC_LEN) { ctx->cdata.key_inline = true; ctx->cdata.key = (uintptr_t)ctx->key; } else { @@ -996,6 +1020,8 @@ static int rfc4106_set_sh_desc(struct crypto_aead *aead) struct device *jrdev = ctx->jrdev; u32 *key_jump_cmd; u32 *desc; + int rem_bytes = CAAM_DESC_BYTES_MAX - GCM_DESC_JOB_IO_LEN - + ctx->cdata.keylen; if (!ctx->cdata.keylen || !ctx->authsize) return 0; @@ -1005,8 +1031,7 @@ static int rfc4106_set_sh_desc(struct crypto_aead *aead) * Job Descriptor and Shared Descriptor * must fit into the 64-word Descriptor h/w Buffer */ - if (DESC_RFC4106_ENC_LEN + GCM_DESC_JOB_IO_LEN + - ctx->cdata.keylen <= CAAM_DESC_BYTES_MAX) { + if (rem_bytes >= DESC_RFC4106_ENC_LEN) { ctx->cdata.key_inline = true; ctx->cdata.key = (uintptr_t)ctx->key; } else { @@ -1084,8 +1109,7 @@ static int rfc4106_set_sh_desc(struct crypto_aead *aead) * Job Descriptor and Shared Descriptors * must all fit into the 64-word Descriptor h/w Buffer */ - if (DESC_RFC4106_DEC_LEN + DESC_JOB_IO_LEN + - ctx->cdata.keylen <= CAAM_DESC_BYTES_MAX) { + if (rem_bytes >= DESC_RFC4106_DEC_LEN) { ctx->cdata.key_inline = true; ctx->cdata.key = (uintptr_t)ctx->key; } else { @@ -1180,6 +1204,8 @@ static int rfc4543_set_sh_desc(struct crypto_aead *aead) u32 *key_jump_cmd; u32 *read_move_cmd, *write_move_cmd; u32 *desc; + int rem_bytes = CAAM_DESC_BYTES_MAX - GCM_DESC_JOB_IO_LEN - + ctx->cdata.keylen; if (!ctx->cdata.keylen || !ctx->authsize) return 0; @@ -1189,8 +1215,7 @@ static int rfc4543_set_sh_desc(struct crypto_aead *aead) * Job Descriptor and Shared Descriptor * must fit into the 64-word Descriptor h/w Buffer */ - if (DESC_RFC4543_ENC_LEN + GCM_DESC_JOB_IO_LEN + - ctx->cdata.keylen <= CAAM_DESC_BYTES_MAX) { + if (rem_bytes >= DESC_RFC4543_ENC_LEN) { ctx->cdata.key_inline = true; ctx->cdata.key = (uintptr_t)ctx->key; } else { @@ -1267,8 +1292,7 @@ static int rfc4543_set_sh_desc(struct crypto_aead *aead) * Job Descriptor and Shared Descriptors * must all fit into the 64-word Descriptor h/w Buffer */ - if (DESC_RFC4543_DEC_LEN + GCM_DESC_JOB_IO_LEN + - ctx->cdata.keylen <= CAAM_DESC_BYTES_MAX) { + if (rem_bytes >= DESC_RFC4543_DEC_LEN) { ctx->cdata.key_inline = true; ctx->cdata.key = (uintptr_t)ctx->key; } else { diff --git a/drivers/crypto/caam/desc_constr.h b/drivers/crypto/caam/desc_constr.h index bfef7952dfb7..fa70c0d79c40 100644 --- a/drivers/crypto/caam/desc_constr.h +++ b/drivers/crypto/caam/desc_constr.h @@ -449,3 +449,42 @@ struct alginfo { u64 key; bool key_inline; }; + +/** + * desc_inline_query() - Provide indications on which data items can be inlined + * and which shall be referenced in a shared descriptor. + * @sd_base_len: Shared descriptor base length - bytes consumed by the commands, + * excluding the data items to be inlined (or corresponding + * pointer if an item is not inlined). Each cnstr_* function that + * generates descriptors should have a define mentioning + * corresponding length. + * @jd_len: Maximum length of the job descriptor(s) that will be used + * together with the shared descriptor. + * @data_len: Array of lengths of the data items trying to be inlined + * @inl_mask: 32bit mask with bit x = 1 if data item x can be inlined, 0 + * otherwise. + * @count: Number of data items (size of @data_len array); must be <= 32 + * + * Return: 0 if data can be inlined / referenced, negative value if not. If 0, + * check @inl_mask for details. + */ +static inline int desc_inline_query(unsigned int sd_base_len, + unsigned int jd_len, unsigned int *data_len, + u32 *inl_mask, unsigned int count) +{ + int rem_bytes = (int)(CAAM_DESC_BYTES_MAX - sd_base_len - jd_len); + unsigned int i; + + *inl_mask = 0; + for (i = 0; (i < count) && (rem_bytes > 0); i++) { + if (rem_bytes - (int)(data_len[i] + + (count - i - 1) * CAAM_PTR_SZ) >= 0) { + rem_bytes -= data_len[i]; + *inl_mask |= (1 << i); + } else { + rem_bytes -= CAAM_PTR_SZ; + } + } + + return (rem_bytes >= 0) ? 0 : -1; +}