From patchwork Tue Dec 20 05:40:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13077513 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4ADC9C4332F for ; Tue, 20 Dec 2022 05:42:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81AD48E0005; Tue, 20 Dec 2022 00:42:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7CA8F8E0001; Tue, 20 Dec 2022 00:42:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 66AE78E0005; Tue, 20 Dec 2022 00:42:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4ED1E8E0001 for ; Tue, 20 Dec 2022 00:42:03 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 626E81C635D for ; Tue, 20 Dec 2022 05:42:02 +0000 (UTC) X-FDA: 80261588484.16.8621C20 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf12.hostedemail.com (Postfix) with ESMTP id CB19740002 for ; Tue, 20 Dec 2022 05:42:00 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=IvUk8ADq; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf12.hostedemail.com: domain of ebiggers@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=ebiggers@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671514921; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j4bxzdAiF9gK3vn6VN8DUzGdwXvIyPjsQRBc11VRhU0=; b=59IFu1T4KN/RDcobniiCLr+S7eFtouxQCdh8tc3ZYid3PA/FMdhkpcdCxf0uFL1aLgfG/K qu6v7q3RCdNw0+WlKf4f25HLkjLxxz3q1kfg9hpzqoSuVb87fWSxT79M6H2dt0xe0c7NEE 4C/CsKW5+R6ICEOibQrjC4y0R+SJL5A= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=IvUk8ADq; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf12.hostedemail.com: domain of ebiggers@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=ebiggers@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671514921; a=rsa-sha256; cv=none; b=IpREG98VNy4+aBXX8flzBpKZZMHJj0nOqwfKeq5XpL5tffmVuk1w8mf9lJw+sQGCq6giGh yabcYXpUlaDtX2dfY33SyJxohvLX2neCjLAwTim//OxOLcl0OZ9q0yKF8jQ8kBFO19zJYP tbziE8eFTj3pe6jWfpPGZy/8RFS5POI= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id D78A2B81196; Tue, 20 Dec 2022 05:41:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6F4F8C43398; Tue, 20 Dec 2022 05:41:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671514917; bh=3n9ADRO4nwA4vng1EWokN1HkHn+cKE/ZleCXkk6FZuU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IvUk8ADqzbVfp/roQjX87ypPf+5oAlzPl/WmfbZwTC7hJGifzKIvrISekXo1QbPwj 77veN4ECatl/Cp4R0djvSgbZtgQ55keX8ZIkZI6jwUwT7Rr+rlabmPl5waJUVMEYj6 XyAo0xmS282IUjrk6asDmm9DdzzAFQgTRU4HxJjUaPmxA1hb+Iu5q+ddQU8JBD6mm3 pmfhJ238h55NLqoCVkky8COjAQAbUY+kKd3+uo5qVrJratiTxIXlJhqeLW1lo+OA7M 9buDvhj1+x2uw3hIpNFy1RbrfxJv0tFYb3c95M8thLEx4T6NY9TZm8pqO+xvypQF+X XOzOZKCnjxGzQ== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: Peter Zijlstra , corbet@lwn.net, will@kernel.org, boqun.feng@gmail.com, mark.rutland@arm.com, catalin.marinas@arm.com, dennis@kernel.org, tj@kernel.org, cl@linux.com, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, Herbert Xu , davem@davemloft.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, joro@8bytes.org, suravee.suthikulpanit@amd.com, robin.murphy@arm.com, dwmw2@infradead.org, baolu.lu@linux.intel.com, Arnd Bergmann , penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, Andrew Morton , vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-s390@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org Subject: [PATCH 1/3] crypto: x86/ghash - fix unaligned access in ghash_setkey() Date: Mon, 19 Dec 2022 21:40:40 -0800 Message-Id: <20221220054042.188537-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20221220054042.188537-1-ebiggers@kernel.org> References: <20221220054042.188537-1-ebiggers@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: CB19740002 X-Stat-Signature: i7yk7u1x6px6fbytwdo7ejqnn6s1qx8m X-HE-Tag: 1671514920-454958 X-HE-Meta: U2FsdGVkX19nTnURlhsKQrFrIvgzmFzkk4VahlB17/9Rk4lOoNm0kuVxcRUilORJFgOYrkLRffvtqRgYBsRJQaq7Lq09/zhTye8U64ZsANmFbSCityz28UUw1e1j852rZGfBUf0YLLj6l5AGZzUv6qTeR1m06gv/71aTDUX2jZM2Ifv7RHe65WYib0n/+BoynEjAI8gL4b8FYQTEPIpb29760HgB4Gjmsahzz9FiygmVJYMtqq70dUQNzzEYQh28SbU8niCtvOEQMW6ACYnHcwODT/E7n5Y/r8txhE6eOcVbtlMEy1x1eSBXk6Tr4YhnIyVFtLE+eOGB+ud8aBedsikT/ngla9OUFS44lNGrdbIxGKIsBmiUwS9ddoGBFqXERG4h83u5ksfxGodpI/JwsWciXk0xnk6shlpHPXHD8GNgai3FiHyzyoRHqYjMuj01zGWHGqDtAreOab8+WrJAzJ70Cq1BGRMpXx0nTQ21t1a9ep+Ji8tk0ool3L2+QsABJcbCOaYL2dcN96K14c5aAEa8sLRkgw7BKSN1CUJSQQdbiqEqSVczuznmUIPuFXXQ4Zha4zEJOU6m5SJnQxxT8fnUxwRuri9UBTUcJ41ueRK9S8JtopNay8inphnWa/1Hbzt/EPCeiqdnwo3a7QJCunV3X3uPW0GD9XJcede1X/Mr+RcsrSEoQjxBpv0awKzCXjHIkhFTjeeGCp3V/gGZ+s7QbLYirhbTNpZRYhoYrRXOHMR/Gof8RhAMcMMxC4BFcar7dlEg03JGIu76eb1Cp2xsJUyidFm6/NLCc1AbPgeH+BNdwPnrO0z49jJEXktG61Hohg4sv4qQ3/3ulQLoXePpPbTErLf4m22Ulim/9UmQSD+US0R52yv1294OY4QEgbrHEj7vGaZsV17785bu/+xoHJOKazmi/8xWJLefNzbXAC/Fqw5DSbe955FDCxneWpgq/FEqNW2XO7L2yDo zAS4/O5P xg4Q/b+FkyMVJIKP1i2WGiXdElN5sjEImTDxifTWSpvOQIRkL11saP12YIntwopWsBgldmF/cvcGmcMWsBhNTBPY6UOKqjRsw2luHQT3fTHfCzg3VMlRuaC9vJqx4siG5oriEwXeudTyrUObQAAw4SZjgBz2zM9iwx/6Hqv55pYWaxBPn5ZB91E5NscsWH83QAW4e X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Eric Biggers The key can be unaligned, so use the unaligned memory access helpers. Fixes: 8ceee72808d1 ("crypto: ghash-clmulni-intel - use C implementation for setkey()") Signed-off-by: Eric Biggers --- arch/x86/crypto/ghash-clmulni-intel_glue.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/x86/crypto/ghash-clmulni-intel_glue.c b/arch/x86/crypto/ghash-clmulni-intel_glue.c index 1f1a95f3dd0c..c0ab0ff4af65 100644 --- a/arch/x86/crypto/ghash-clmulni-intel_glue.c +++ b/arch/x86/crypto/ghash-clmulni-intel_glue.c @@ -19,6 +19,7 @@ #include #include #include +#include #define GHASH_BLOCK_SIZE 16 #define GHASH_DIGEST_SIZE 16 @@ -54,15 +55,14 @@ static int ghash_setkey(struct crypto_shash *tfm, const u8 *key, unsigned int keylen) { struct ghash_ctx *ctx = crypto_shash_ctx(tfm); - be128 *x = (be128 *)key; u64 a, b; if (keylen != GHASH_BLOCK_SIZE) return -EINVAL; /* perform multiplication by 'x' in GF(2^128) */ - a = be64_to_cpu(x->a); - b = be64_to_cpu(x->b); + a = get_unaligned_be64(key); + b = get_unaligned_be64(key + 8); ctx->shash.a = (b << 1) | (a >> 63); ctx->shash.b = (a << 1) | (b >> 63); From patchwork Tue Dec 20 05:40:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13077512 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64FD7C4167B for ; Tue, 20 Dec 2022 05:42:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BEA278E0003; Tue, 20 Dec 2022 00:42:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B9BC18E0001; Tue, 20 Dec 2022 00:42:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3AB98E0003; Tue, 20 Dec 2022 00:42:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 8FB7D8E0001 for ; Tue, 20 Dec 2022 00:42:02 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 62388A0A86 for ; Tue, 20 Dec 2022 05:42:02 +0000 (UTC) X-FDA: 80261588484.13.0C0E983 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf15.hostedemail.com (Postfix) with ESMTP id B8772A0006 for ; Tue, 20 Dec 2022 05:42:00 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=b4QPIeBD; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf15.hostedemail.com: domain of ebiggers@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=ebiggers@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671514921; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9RW0us+nStEgHZTGP1VwlpkJJVA72mMv6v798BL7tQ8=; b=PI+JVYiRAjvH8enDcJ5UcFF7JUcNx7Bn70fRO/45t0vU+RiaOecx4QdsYaDDZK+gUsuicu MPYnlZ5XTYLPte5BCVAb0rN3ZksoZp/p0fTv7sLb2rfAt3B0KMJRgyIO8AsLuTyC5asFAW RHdx82pg2A0ZPWE2TSlo66sM+C8axhs= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=b4QPIeBD; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf15.hostedemail.com: domain of ebiggers@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=ebiggers@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671514921; a=rsa-sha256; cv=none; b=2R8sF0uQZkNLrkr1z48BaHUdbvbbqq6reqSBqc/5Z6gh8VXRUGxw5w173K8RJ3orPi/8vJ jHCkMks6BufWJVLO8eMaxIpRaq4jjg/igT8iF/o1LzQFwpu8dWR/u9wvG7/GdkI0aTPrND yRvsGfb3LEq5MavX73V/5WxFpr0qiqw= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 5B064B811AC; Tue, 20 Dec 2022 05:41:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C8630C433D2; Tue, 20 Dec 2022 05:41:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671514919; bh=r7CcLqy65ywV0RdmBdnF4gvrkE0V7pqcwnuVzqA286k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b4QPIeBDySeIw45tMI1nPwu6/LnNHxYWm2KdE0KmbJJA+QQ5NSMnGSgSTskJe068B a4O19R9sThYWaWHTRdN875o8NAIGdCht29RqD8fd3R9QJl5M5nPP2k0Qz/J13j+q9R szrJdxF/p54c3WPQPSOPRTgNaRp3zI0yGR3/pWPy5cZrzPmvfYUoTj8vlTrc+KfrLo r90Npm4NnbnGF2uk5qR5byeWzPI4ZzJhJp+zuPusgSYrt/dkFyuwP14LiGBH1ZF6YB FY0+pzVMYYmqqbGAPXLoFNY7XaYJRvV9podM+CJY4nTz7ktpEaNJDoh1F1ookspoH4 H6ZBz8YDu0PEw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: Peter Zijlstra , corbet@lwn.net, will@kernel.org, boqun.feng@gmail.com, mark.rutland@arm.com, catalin.marinas@arm.com, dennis@kernel.org, tj@kernel.org, cl@linux.com, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, Herbert Xu , davem@davemloft.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, joro@8bytes.org, suravee.suthikulpanit@amd.com, robin.murphy@arm.com, dwmw2@infradead.org, baolu.lu@linux.intel.com, Arnd Bergmann , penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, Andrew Morton , vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-s390@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org Subject: [PATCH 2/3] crypto: x86/ghash - use le128 instead of u128 Date: Mon, 19 Dec 2022 21:40:41 -0800 Message-Id: <20221220054042.188537-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20221220054042.188537-1-ebiggers@kernel.org> References: <20221220054042.188537-1-ebiggers@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: B8772A0006 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: ysjq7kd34bkmnaqhf7r48shuw9pkitog X-HE-Tag: 1671514920-918319 X-HE-Meta: U2FsdGVkX198jZyRN4ZmpW/8ytujtKrh0uKmCrFWC9nmj8pbu96EhTyxdN8Uht/wD55j3fMmsQGV5rC6oNyfDsuDBhnm4775RjCDVKRO2TapHhDhfKntMrsN9lE7qKiYKnkk6pPyKvFwBwusfFZmPqr6E145Bv7wKLlIbfllCRBd3zSdh9v9Lv4VArI15qJ1/wfJ/1i0DUEhhdMTXkqAz9yu6vFn0klidpo5u66ZA3OEwhJVEeHQ+/cl0wPgda7wvIJZEMB/d0f8W3e07W2isw2fq1WTIQl+Pcln4FmOT3mxPG6Oy4Sl6lEP96GZdHMn4O03i4BGfwxi2cZPYVuBosIHEEDVzNS5mPqSpbavNKQhK741xqecMUM8dZqOcoPdJ74NqAZ6O0Ab1X+2dRmRaWs1m3zxPpc8iHzLDoTIU/NGzUUiZAIBeD5WS+CXWEP7GPf+9GYuvLuf3IxkvSyq3UBolnuZJNB8BwICCrksrwwNqw1iCPBwdklcC3taJ593pa4/xAzQx6QcvW2RwbD/WA2jWQ2huZwJpwRr92/H8jCQRoM0ew2Uaz+54pfyAHwzpbk4nZxYLY7BmokFhyf+AIwzkQufZUihtge2NqDVLlqwgavUFf1h4/WHg8Km6Y5U88Das8p7MutsJrcYIoqbgMufpLRdK5F1K9bf3UNg5fWJJSI32H/91ci5GvWr3d6SZMYQ0VectfwWfnvDnY5E7z18e0h9nMxlVAbUhhiZbyRJZW4kEmCFWkb9L3FIBFDauwNf8X9gnc7Q6D3nhsmPkaFHESUrSvOZ9OSynhUUSwwdrCPFsFTdpCOgsNkPT5qK36B6M07wMxNNiuRfxlG71JWgi0m/S4xTkaqDP1k2RyrjhDCYeO47hbUZgMlcU4YTDgE9cGhwfq4BeZw/zfmo60ZNSdOOGeyPlmwh5i/NDg/BZ/VBP27BolUzTweQcwezMrGPzTn5z16U8URFqIh 5SphPPWm 4sgnDddh/iR56b7FKyY/vBpUFTQASJVIn4b3/ZMtjBrd2dC6NkGUK4FOUZZ9RosgqrGzDBySjWbIOfVsp9CC4hZ3gnWBFQTh/qJaUQ1BVD2fkCnV4nAhBLjDt9IHdygEt1qBNUQiY4zp2m48= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Eric Biggers The u128 struct type is going away, so make ghash-clmulni-intel use le128 instead. Note that the field names a and b swapped, as they were backwards with u128. (a is meant to be high-order and b low-order.) Signed-off-by: Eric Biggers --- arch/x86/crypto/ghash-clmulni-intel_asm.S | 4 ++-- arch/x86/crypto/ghash-clmulni-intel_glue.c | 12 ++++++------ 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/x86/crypto/ghash-clmulni-intel_asm.S b/arch/x86/crypto/ghash-clmulni-intel_asm.S index 2bf871899920..9dfeb4d31b92 100644 --- a/arch/x86/crypto/ghash-clmulni-intel_asm.S +++ b/arch/x86/crypto/ghash-clmulni-intel_asm.S @@ -88,7 +88,7 @@ SYM_FUNC_START_LOCAL(__clmul_gf128mul_ble) RET SYM_FUNC_END(__clmul_gf128mul_ble) -/* void clmul_ghash_mul(char *dst, const u128 *shash) */ +/* void clmul_ghash_mul(char *dst, const le128 *shash) */ SYM_FUNC_START(clmul_ghash_mul) FRAME_BEGIN movups (%rdi), DATA @@ -104,7 +104,7 @@ SYM_FUNC_END(clmul_ghash_mul) /* * void clmul_ghash_update(char *dst, const char *src, unsigned int srclen, - * const u128 *shash); + * const le128 *shash); */ SYM_FUNC_START(clmul_ghash_update) FRAME_BEGIN diff --git a/arch/x86/crypto/ghash-clmulni-intel_glue.c b/arch/x86/crypto/ghash-clmulni-intel_glue.c index c0ab0ff4af65..9453b094bb3b 100644 --- a/arch/x86/crypto/ghash-clmulni-intel_glue.c +++ b/arch/x86/crypto/ghash-clmulni-intel_glue.c @@ -24,17 +24,17 @@ #define GHASH_BLOCK_SIZE 16 #define GHASH_DIGEST_SIZE 16 -void clmul_ghash_mul(char *dst, const u128 *shash); +void clmul_ghash_mul(char *dst, const le128 *shash); void clmul_ghash_update(char *dst, const char *src, unsigned int srclen, - const u128 *shash); + const le128 *shash); struct ghash_async_ctx { struct cryptd_ahash *cryptd_tfm; }; struct ghash_ctx { - u128 shash; + le128 shash; }; struct ghash_desc_ctx { @@ -64,11 +64,11 @@ static int ghash_setkey(struct crypto_shash *tfm, a = get_unaligned_be64(key); b = get_unaligned_be64(key + 8); - ctx->shash.a = (b << 1) | (a >> 63); - ctx->shash.b = (a << 1) | (b >> 63); + ctx->shash.a = cpu_to_le64((a << 1) | (b >> 63)); + ctx->shash.b = cpu_to_le64((b << 1) | (a >> 63)); if (a >> 63) - ctx->shash.b ^= ((u64)0xc2) << 56; + ctx->shash.a ^= cpu_to_le64((u64)0xc2 << 56); return 0; } From patchwork Tue Dec 20 05:40:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13077514 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2050C10F1E for ; Tue, 20 Dec 2022 05:42:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 362FB8E0006; Tue, 20 Dec 2022 00:42:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2E7928E0001; Tue, 20 Dec 2022 00:42:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 188798E0006; Tue, 20 Dec 2022 00:42:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 0218B8E0001 for ; Tue, 20 Dec 2022 00:42:04 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id C08B580316 for ; Tue, 20 Dec 2022 05:42:03 +0000 (UTC) X-FDA: 80261588526.05.AC96FD6 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf29.hostedemail.com (Postfix) with ESMTP id 0C5D1120009 for ; Tue, 20 Dec 2022 05:42:01 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jYOojsuB; spf=pass (imf29.hostedemail.com: domain of ebiggers@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=ebiggers@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671514922; a=rsa-sha256; cv=none; b=8Kzx3mf3Oy0BgtU4WFaz0QaUKYPjCgICoASvlVTqp4jJJvOtjgs66a+iHJwBIPaJ9ZmQAy bjvVA5hue2Z8WnH8Yjxvtd8BPiaWZYMlrPtq3FK5FBijCgQZ+/WQdbg+sETt6dlw6yTx0F JVLBH01QZZqj2VPbXRGoMUJojtcjefc= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jYOojsuB; spf=pass (imf29.hostedemail.com: domain of ebiggers@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=ebiggers@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671514922; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nOYXpDiFyQEKhsLgatHAwSyheaaratVquM7fRx8vRaI=; b=pcCLxUWyRxPDBQsEVsw7EIfPTuLCo3JDozjn3qKc3K85Y2KoOXUvuuB3HAMjq1HG0Neg77 VR9LHO9ZtrmEU/+0gz93RWcE1Nnbf3SXlrqp/XnjJoEqYQWqfWGBTZz8t6f/hLVUim1RBP lu6ycZ62C/NOaQSEFJVXV+oRIqFSKc8= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B0934B811AF; Tue, 20 Dec 2022 05:42:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3059AC433A0; Tue, 20 Dec 2022 05:41:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671514920; bh=dsBT80rMbHb7evBtDRzv5PT3ZA7g7MTpg/OO26ExzF0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jYOojsuBCSLHGr8TW6ECUwgWw+LEvw+PHi3nql2vBWzBzEq7MyKtVOLJ/b2Kiu4gS oMG/dlg3OnMmVJmrdSv2oL97DF/CUUglF7nDPoQjrR1EUjWMzPIKMXZdHUPv8l8Sjd UXS+WUfMJ0NGEcylNFSYCWZt2yo1UQq4aV3vPbfVoQuD+Qp8TbeWtUxBsxReFPOxlA zfs32JevZlz+K+Uv5duClJslwi2mYsu29ttlLiM4znIDXOp4GXmEIq6usK1oiCT/rX LwdwbARx6KolYJUmb/r6mqXzDaBmbWq87tBtJKhhILBQJc9wo5CJtT/46YobC23TXN w7YoOtqqwQbVA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: Peter Zijlstra , corbet@lwn.net, will@kernel.org, boqun.feng@gmail.com, mark.rutland@arm.com, catalin.marinas@arm.com, dennis@kernel.org, tj@kernel.org, cl@linux.com, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, Herbert Xu , davem@davemloft.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, joro@8bytes.org, suravee.suthikulpanit@amd.com, robin.murphy@arm.com, dwmw2@infradead.org, baolu.lu@linux.intel.com, Arnd Bergmann , penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, Andrew Morton , vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-s390@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org Subject: [PATCH 3/3] crypto: x86/ghash - add comment and fix broken link Date: Mon, 19 Dec 2022 21:40:42 -0800 Message-Id: <20221220054042.188537-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20221220054042.188537-1-ebiggers@kernel.org> References: <20221220054042.188537-1-ebiggers@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 0C5D1120009 X-Rspamd-Server: rspam01 X-Stat-Signature: nos71t6mkzsscrkpjntz97o8tmbd4hmu X-HE-Tag: 1671514921-493362 X-HE-Meta: U2FsdGVkX1/bDv+2ElGHaLsgswQudgur9T46DTVuGxNELQA0Wfp+osNo8qlAkGahsa2yq6yAmXu0NXO4tvQh00/04fgUVaKcIxqWsiT+vC/j116bPzb7H/30JumC3oimEwaxxzXobX9SezjZJnMlSqZ6kxCF8WHsBPx+isrBgJNxtdMVkBwuZNyq9+nCESDdkxn4zO8B151Zciw8PalElBDTov1VOXHHC4KFBNXwTHgBeedqAbBfHwJ8OFKFq1HYjsS8CXKk8WwkDt9MLnDxRiZgsHTRsOQKfHQjI1ZKVAWZ25meAQv91i5AVTk85p/Crvt0zfg/IpGUAizZBjU5+nScKyiTuDfxq38D6r3Fon3m7BA8r/RFDrPhl6d/brE+Zsmv+LI1CAxsMi/MaDf/fe5uXB7Ap49o+IndGmoVrnILZxX8De48AlkFNyg+8F6nxe9SREB/JDC7Fi0UbymmMbpo12WT+gBA+7mCgTg8aki+Vf9Y3uin00D8YWgXov8IVmGEpTdzUc02EJXgA1aOo4bFKXL/5kCFaOpZbWRAfPjvLZFA7a03bh56ul8VecCfABlhoA8gW2UqHEvBWfG9678qU8Bw0RmCqVDZPlr6koxTb7GInm3WRTkJGCyXWQUiZ2rb+QEv03Pvu60gKh55BKrM5H/R6bG7C93ecUvB5N3VruVl8btX2fWgY8O9RMs/wINncQo0OMYSHyvfR/aifylbdGm1QkJphaRX/BOhJwyQFLNm05+3wj4EousPU7gLbItZhe/NkETdioTf8UkiN2U+iCp56c0FSz7emKhbBBtugoA5MJQBtG0577ijcdZPmTtCdZGqkS2PezVe4Gz4R6tSqti3da3pC/GX3iIwK2bKNpntxCl5WhRktLcRrXNTgKCBbIJA8ML+WVcDJshd0XtsumYSY0Vk+z2elWV6yEJPhk7BXUY52wH7Rw0+qsyzE/t5MG78jz+kSBzeskA TgVKkZkx MUL3C4rtS8+2AZck2m2O6bW/LA8BQK+k3FOyHU/tLohP/gF3Abh+5vuTGzxATJZKe1wKQe7JtJAUprEk2x4p5VHpK8WLpQpbLetPYKq4SjVP+4RbZSwSZAGaP6Hs8L1pFO7tLBj4TgzBAS3Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Eric Biggers Add a comment that explains what ghash_setkey() is doing, as it's hard to understand otherwise. Also fix a broken hyperlink. Signed-off-by: Eric Biggers --- arch/x86/crypto/ghash-clmulni-intel_asm.S | 2 +- arch/x86/crypto/ghash-clmulni-intel_glue.c | 27 ++++++++++++++++++---- 2 files changed, 24 insertions(+), 5 deletions(-) diff --git a/arch/x86/crypto/ghash-clmulni-intel_asm.S b/arch/x86/crypto/ghash-clmulni-intel_asm.S index 9dfeb4d31b92..257ed9446f3e 100644 --- a/arch/x86/crypto/ghash-clmulni-intel_asm.S +++ b/arch/x86/crypto/ghash-clmulni-intel_asm.S @@ -4,7 +4,7 @@ * instructions. This file contains accelerated part of ghash * implementation. More information about PCLMULQDQ can be found at: * - * http://software.intel.com/en-us/articles/carry-less-multiplication-and-its-usage-for-computing-the-gcm-mode/ + * https://www.intel.com/content/dam/develop/external/us/en/documents/clmul-wp-rev-2-02-2014-04-20.pdf * * Copyright (c) 2009 Intel Corp. * Author: Huang Ying diff --git a/arch/x86/crypto/ghash-clmulni-intel_glue.c b/arch/x86/crypto/ghash-clmulni-intel_glue.c index 9453b094bb3b..700ecaee9a08 100644 --- a/arch/x86/crypto/ghash-clmulni-intel_glue.c +++ b/arch/x86/crypto/ghash-clmulni-intel_glue.c @@ -60,16 +60,35 @@ static int ghash_setkey(struct crypto_shash *tfm, if (keylen != GHASH_BLOCK_SIZE) return -EINVAL; - /* perform multiplication by 'x' in GF(2^128) */ + /* + * GHASH maps bits to polynomial coefficients backwards, which makes it + * hard to implement. But it can be shown that the GHASH multiplication + * + * D * K (mod x^128 + x^7 + x^2 + x + 1) + * + * (where D is a data block and K is the key) is equivalent to: + * + * bitreflect(D) * bitreflect(K) * x^(-127) + * (mod x^128 + x^127 + x^126 + x^121 + 1) + * + * So, the code below precomputes: + * + * bitreflect(K) * x^(-127) (mod x^128 + x^127 + x^126 + x^121 + 1) + * + * ... but in Montgomery form (so that Montgomery multiplication can be + * used), i.e. with an extra x^128 factor, which means actually: + * + * bitreflect(K) * x (mod x^128 + x^127 + x^126 + x^121 + 1) + * + * The within-a-byte part of bitreflect() cancels out GHASH's built-in + * reflection, and thus bitreflect() is actually a byteswap. + */ a = get_unaligned_be64(key); b = get_unaligned_be64(key + 8); - ctx->shash.a = cpu_to_le64((a << 1) | (b >> 63)); ctx->shash.b = cpu_to_le64((b << 1) | (a >> 63)); - if (a >> 63) ctx->shash.a ^= cpu_to_le64((u64)0xc2 << 56); - return 0; }