From patchwork Mon Jan 27 22:38:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13951801 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A1E481D95B3; Mon, 27 Jan 2025 22:38:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738017538; cv=none; b=LWynrdcGq4PUdXaPr5C+sP9Ie21uy7LMvUvvNLE5fNMHTKzoVy0VL9n0ro6UZdozs5t6NgybggO0SwnAzD+iKghi2EERAF6jlRcNkbfGiXhbgPlRmNiK7XEKVbVmacOSqpA87AyteoGrRfTHCuBBgDrXtroeBNyvjIrI4UkiBYk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738017538; c=relaxed/simple; bh=AdUG2oG1qWlGDUzaatWr141MRlZ6IfJJpmCsLymetus=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=r6k80vRHc8gfCMP2NoSniJRhAnSZxMGOguyJk6MTv+68bWDEGwjU0a3jKXbDl1BiqIwrZJhqgYEJiBuc1AoZ/bWI9b6juuy2G8CoEh/tryy6GcyEe06QHlmMexKpDHOMnVNduvHf1w2MjmECfF3XUAB9zM/ABpWV0vlRe9JXHNU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=N+yCrWH6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="N+yCrWH6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 474CEC4CEE3; Mon, 27 Jan 2025 22:38:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738017538; bh=AdUG2oG1qWlGDUzaatWr141MRlZ6IfJJpmCsLymetus=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=N+yCrWH62yygIT+MZf+sWNXNQ+kc9jshsP9MDjPCcCnWSQR/SKjnRBzlYCsQgNKVV K+ja92LfDab3WI8WFWxSTUd2MjIvt1t51k0fcTs691vQe+SKVtZkFDMknGTXtvrcrJ 0dci7u7xqC/HLmkH9/AkdGQ1RdtYOxN4doQx9lzygIRJEx3ppT4Ke0lWe7BjMJp6/z slJCegPmwGno2WqLxqtHXgZUme9bTBmjFNm5C3c7Lc4yZ1c76deQxNRCRSbdRNr9IV Hoplmx4sN+e/HQdIi0x1CdYLrBsW3qXc7OMsKKGQsOURuwIB6TNF5HvALWeWZcrZrF H++/uenKNSKkA== From: Eric Biggers To: linux-rdma@vger.kernel.org, Mustafa Ismail , Tatyana Nikolova , Jason Gunthorpe , Leon Romanovsky , Zhu Yanjun , Bernard Metzler Cc: linux-kernel@vger.kernel.org Subject: [PATCH 1/6] RDMA/rxe: handle ICRC correctly on big endian systems Date: Mon, 27 Jan 2025 14:38:35 -0800 Message-ID: <20250127223840.67280-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250127223840.67280-1-ebiggers@kernel.org> References: <20250127223840.67280-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Eric Biggers The usual big endian convention of InfiniBand does not apply to the ICRC field, whose transmission is specified in terms of the CRC32 polynomial coefficients. The coefficients are transmitted in descending order from the x^31 coefficient to the x^0 one. When the result is interpreted as a 32-bit integer using the required reverse mapping between bits and polynomial coefficients, it's a __le32. The code used __be32, but it did not actually do an endianness conversion. So it actually just used the CPU's native endianness, making it comply with the spec on little endian systems only. Fix the code to use __le32 and do the needed le32_to_cpu() and cpu_to_le32() so that it complies with the spec on big endian systems. Also update the lower-level helper functions to use u32, as they are working with CPU native values. This part does not change behavior. Found through code review. I don't know whether anyone is actually using this code on big endian systems. Probably not. Signed-off-by: Eric Biggers Reviewed-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe_icrc.c | 41 +++++++++++++++------------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_icrc.c b/drivers/infiniband/sw/rxe/rxe_icrc.c index fdf5f08cd8f17..ee417a0bbbdca 100644 --- a/drivers/infiniband/sw/rxe/rxe_icrc.c +++ b/drivers/infiniband/sw/rxe/rxe_icrc.c @@ -38,26 +38,26 @@ int rxe_icrc_init(struct rxe_dev *rxe) * @next: starting address of current segment * @len: length of current segment * * Return: the cumulative crc32 checksum */ -static __be32 rxe_crc32(struct rxe_dev *rxe, __be32 crc, void *next, size_t len) +static u32 rxe_crc32(struct rxe_dev *rxe, u32 crc, void *next, size_t len) { - __be32 icrc; + u32 icrc; int err; SHASH_DESC_ON_STACK(shash, rxe->tfm); shash->tfm = rxe->tfm; - *(__be32 *)shash_desc_ctx(shash) = crc; + *(u32 *)shash_desc_ctx(shash) = crc; err = crypto_shash_update(shash, next, len); if (unlikely(err)) { rxe_dbg_dev(rxe, "failed crc calculation, err: %d\n", err); - return (__force __be32)crc32_le((__force u32)crc, next, len); + return crc32_le(crc, next, len); } - icrc = *(__be32 *)shash_desc_ctx(shash); + icrc = *(u32 *)shash_desc_ctx(shash); barrier_data(shash_desc_ctx(shash)); return icrc; } @@ -67,18 +67,18 @@ static __be32 rxe_crc32(struct rxe_dev *rxe, __be32 crc, void *next, size_t len) * @skb: packet buffer * @pkt: packet information * * Return: the partial ICRC */ -static __be32 rxe_icrc_hdr(struct sk_buff *skb, struct rxe_pkt_info *pkt) +static u32 rxe_icrc_hdr(struct sk_buff *skb, struct rxe_pkt_info *pkt) { unsigned int bth_offset = 0; struct iphdr *ip4h = NULL; struct ipv6hdr *ip6h = NULL; struct udphdr *udph; struct rxe_bth *bth; - __be32 crc; + u32 crc; int length; int hdr_size = sizeof(struct udphdr) + (skb->protocol == htons(ETH_P_IP) ? sizeof(struct iphdr) : sizeof(struct ipv6hdr)); /* pseudo header buffer size is calculate using ipv6 header size since @@ -89,11 +89,11 @@ static __be32 rxe_icrc_hdr(struct sk_buff *skb, struct rxe_pkt_info *pkt) RXE_BTH_BYTES]; /* This seed is the result of computing a CRC with a seed of * 0xfffffff and 8 bytes of 0xff representing a masked LRH. */ - crc = (__force __be32)0xdebb20e3; + crc = 0xdebb20e3; if (skb->protocol == htons(ETH_P_IP)) { /* IPv4 */ memcpy(pshdr, ip_hdr(skb), hdr_size); ip4h = (struct iphdr *)pshdr; udph = (struct udphdr *)(ip4h + 1); @@ -137,23 +137,27 @@ static __be32 rxe_icrc_hdr(struct sk_buff *skb, struct rxe_pkt_info *pkt) * * Return: 0 if the values match else an error */ int rxe_icrc_check(struct sk_buff *skb, struct rxe_pkt_info *pkt) { - __be32 *icrcp; - __be32 pkt_icrc; - __be32 icrc; - - icrcp = (__be32 *)(pkt->hdr + pkt->paylen - RXE_ICRC_SIZE); - pkt_icrc = *icrcp; + /* + * The usual big endian convention of InfiniBand does not apply to the + * ICRC field, whose transmission is specified in terms of the CRC32 + * polynomial coefficients. The coefficients are transmitted in + * descending order from the x^31 coefficient to the x^0 one. When the + * result is interpreted as a 32-bit integer using the required reverse + * mapping between bits and polynomial coefficients, it's a __le32. + */ + __le32 *icrcp = (__le32 *)(pkt->hdr + pkt->paylen - RXE_ICRC_SIZE); + u32 icrc; icrc = rxe_icrc_hdr(skb, pkt); icrc = rxe_crc32(pkt->rxe, icrc, (u8 *)payload_addr(pkt), payload_size(pkt) + bth_pad(pkt)); icrc = ~icrc; - if (unlikely(icrc != pkt_icrc)) + if (unlikely(icrc != le32_to_cpu(*icrcp))) return -EINVAL; return 0; } @@ -162,14 +166,13 @@ int rxe_icrc_check(struct sk_buff *skb, struct rxe_pkt_info *pkt) * @skb: packet buffer * @pkt: packet information */ void rxe_icrc_generate(struct sk_buff *skb, struct rxe_pkt_info *pkt) { - __be32 *icrcp; - __be32 icrc; + __le32 *icrcp = (__le32 *)(pkt->hdr + pkt->paylen - RXE_ICRC_SIZE); + u32 icrc; - icrcp = (__be32 *)(pkt->hdr + pkt->paylen - RXE_ICRC_SIZE); icrc = rxe_icrc_hdr(skb, pkt); icrc = rxe_crc32(pkt->rxe, icrc, (u8 *)payload_addr(pkt), payload_size(pkt) + bth_pad(pkt)); - *icrcp = ~icrc; + *icrcp = cpu_to_le32(~icrc); } From patchwork Mon Jan 27 22:38:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13951803 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 76EC41DC99A; Mon, 27 Jan 2025 22:38:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738017539; cv=none; b=LA3z9e+gNSdrg99QWOIiL26PSAxh8Ek02wtivSxl9prtnLYm7eOJhQn8nGECGnYpYAK92RI01FpibpdIpQK1++PZrahjmjWnovUet9ryM17G/fluSFv3w78oUYTFnh/TEtQuxp8acHjIwHvSKE3RXQc8J+jC6BDOwF+Suclujec= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738017539; c=relaxed/simple; bh=RcffRuLM1HcK81UB5AITUqZeaIYml6UqCkQoXQ0R/Lw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KRRinbfgiBHHVc+4fvQFpnQJd5CRDbUxjdPCAOPJbZ/YGkQXtAIh+mACratFL5xd6iiCvvcdr+WilwbOdGXkLNaQ3PJDiUWUNfJJTnLehsWxwQt1dklo8RW4uJ5CwmdHaXY2jfJPQOTL79HcgiOjAT5Ft5gP5iZtK0jgFLVo48w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=aM12lIaF; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="aM12lIaF" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A519AC4CEE2; Mon, 27 Jan 2025 22:38:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738017538; bh=RcffRuLM1HcK81UB5AITUqZeaIYml6UqCkQoXQ0R/Lw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aM12lIaF7g51vEvycs/hO5JriSdeKel+inG/QqQYM/lxBGeIegY3iff2e8ahI3TA0 X9NbDFYayxRL7Dmp9eIxqxBiWSeW1yKsBATj9ixhY+5XTUwNyPZ/St5sczLBiRpYGg 7IgOqh4yj+H1IxSCv+wvHMATRWB40gsTcGn0Lgx1n7QoDlEyvAv0W30W5Xfp2uUeQ0 B6E9+VXcpM6nAje+ejOaNsDc/3NoMzqKy/at4NfZsfBdc0n2gMqUZ0hJUiF4pBrA7h hn6a+8Y0GbK7X4mYwytyAn3e3pQEKTkVnikK/ZDdTYTevZxWJirQy7iz0PZ/JC+8q5 xjQrFtWLRAK+A== From: Eric Biggers To: linux-rdma@vger.kernel.org, Mustafa Ismail , Tatyana Nikolova , Jason Gunthorpe , Leon Romanovsky , Zhu Yanjun , Bernard Metzler Cc: linux-kernel@vger.kernel.org Subject: [PATCH 2/6] RDMA/rxe: consolidate code for calculating ICRC of packets Date: Mon, 27 Jan 2025 14:38:36 -0800 Message-ID: <20250127223840.67280-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250127223840.67280-1-ebiggers@kernel.org> References: <20250127223840.67280-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Eric Biggers Since rxe_icrc_hdr() is always immediately followed by updating the CRC with the packet's payload, just rename it to rxe_icrc() and make it include the payload in the CRC, so it now handles the entire packet. This is a refactor with no change in behavior. Signed-off-by: Eric Biggers Reviewed-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe_icrc.c | 36 ++++++++++++---------------- 1 file changed, 15 insertions(+), 21 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_icrc.c b/drivers/infiniband/sw/rxe/rxe_icrc.c index ee417a0bbbdca..c7b0b4673b959 100644 --- a/drivers/infiniband/sw/rxe/rxe_icrc.c +++ b/drivers/infiniband/sw/rxe/rxe_icrc.c @@ -60,26 +60,24 @@ static u32 rxe_crc32(struct rxe_dev *rxe, u32 crc, void *next, size_t len) return icrc; } /** - * rxe_icrc_hdr() - Compute the partial ICRC for the network and transport - * headers of a packet. + * rxe_icrc() - Compute the ICRC of a packet * @skb: packet buffer * @pkt: packet information * - * Return: the partial ICRC + * Return: the ICRC */ -static u32 rxe_icrc_hdr(struct sk_buff *skb, struct rxe_pkt_info *pkt) +static u32 rxe_icrc(struct sk_buff *skb, struct rxe_pkt_info *pkt) { unsigned int bth_offset = 0; struct iphdr *ip4h = NULL; struct ipv6hdr *ip6h = NULL; struct udphdr *udph; struct rxe_bth *bth; u32 crc; - int length; int hdr_size = sizeof(struct udphdr) + (skb->protocol == htons(ETH_P_IP) ? sizeof(struct iphdr) : sizeof(struct ipv6hdr)); /* pseudo header buffer size is calculate using ipv6 header size since * it is bigger than ipv4 @@ -118,17 +116,23 @@ static u32 rxe_icrc_hdr(struct sk_buff *skb, struct rxe_pkt_info *pkt) bth = (struct rxe_bth *)&pshdr[bth_offset]; /* exclude bth.resv8a */ bth->qpn |= cpu_to_be32(~BTH_QPN_MASK); - length = hdr_size + RXE_BTH_BYTES; - crc = rxe_crc32(pkt->rxe, crc, pshdr, length); + /* Update the CRC with the first part of the headers. */ + crc = rxe_crc32(pkt->rxe, crc, pshdr, hdr_size + RXE_BTH_BYTES); - /* And finish to compute the CRC on the remainder of the headers. */ + /* Update the CRC with the remainder of the headers. */ crc = rxe_crc32(pkt->rxe, crc, pkt->hdr + RXE_BTH_BYTES, rxe_opcode[pkt->opcode].length - RXE_BTH_BYTES); - return crc; + + /* Update the CRC with the payload. */ + crc = rxe_crc32(pkt->rxe, crc, payload_addr(pkt), + payload_size(pkt) + bth_pad(pkt)); + + /* Invert the CRC and return it. */ + return ~crc; } /** * rxe_icrc_check() - Compute ICRC for a packet and compare to the ICRC * delivered in the packet. @@ -146,18 +150,12 @@ int rxe_icrc_check(struct sk_buff *skb, struct rxe_pkt_info *pkt) * descending order from the x^31 coefficient to the x^0 one. When the * result is interpreted as a 32-bit integer using the required reverse * mapping between bits and polynomial coefficients, it's a __le32. */ __le32 *icrcp = (__le32 *)(pkt->hdr + pkt->paylen - RXE_ICRC_SIZE); - u32 icrc; - icrc = rxe_icrc_hdr(skb, pkt); - icrc = rxe_crc32(pkt->rxe, icrc, (u8 *)payload_addr(pkt), - payload_size(pkt) + bth_pad(pkt)); - icrc = ~icrc; - - if (unlikely(icrc != le32_to_cpu(*icrcp))) + if (unlikely(rxe_icrc(skb, pkt) != le32_to_cpu(*icrcp))) return -EINVAL; return 0; } @@ -167,12 +165,8 @@ int rxe_icrc_check(struct sk_buff *skb, struct rxe_pkt_info *pkt) * @pkt: packet information */ void rxe_icrc_generate(struct sk_buff *skb, struct rxe_pkt_info *pkt) { __le32 *icrcp = (__le32 *)(pkt->hdr + pkt->paylen - RXE_ICRC_SIZE); - u32 icrc; - icrc = rxe_icrc_hdr(skb, pkt); - icrc = rxe_crc32(pkt->rxe, icrc, (u8 *)payload_addr(pkt), - payload_size(pkt) + bth_pad(pkt)); - *icrcp = cpu_to_le32(~icrc); + *icrcp = cpu_to_le32(rxe_icrc(skb, pkt)); } From patchwork Mon Jan 27 22:38:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13951804 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B16F81DC9AD; Mon, 27 Jan 2025 22:38:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738017539; cv=none; b=VYbPhgPEYagDUQv2Fa/eG3uMfd+dmB8aKPXvtc1esRyZ+BcBsrdViZavZO1k5qiEQzyVDFOcynxx63NK1cNjeDiLhPqpseg0MdgQv0I+4SmgHnpwlZb++zuJkXQmid63kB7AjkeWQcO5euoHmagA28qtgpFQoRoREERiO70losQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738017539; c=relaxed/simple; bh=zZV+2pn+ZuAYgEabFNvBWhziQ9Ptxfsx63rwYN2kkGo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mO6Ya/OwFWyJ7BerDRbj7pAohJGQ/wIj4WSh4qAOoUwH75b4xZo1/PPBDTnmcp0onhgTXxDA1oRUfsqhdVge5ws0xg5OOUA7XMii3CtX2ZQXgTRMBvnQvs7rAzCq/6F8Fhn+BgouB+8lKzRT+oxp2VXqFMmqeAJUtl3c86SC5lI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=n4E8sbm1; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="n4E8sbm1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0F2B8C4CEE0; Mon, 27 Jan 2025 22:38:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738017539; bh=zZV+2pn+ZuAYgEabFNvBWhziQ9Ptxfsx63rwYN2kkGo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=n4E8sbm1krbJvndI4bZk45jfdwolfLT41Vw7u4tcnM/+KO3RzD6PNJF/9b4jAD/hz nUuq2E2hk9hppDHA3tnRbQf+HDoc841Wm8KCsn4z/+JC/c1kzLvGoozuiS7cD+lg3F DI23jZ0SPu5zWFpwoIT+9173/tdVigIeJzt/tibwJAA/8FdIbuGE8gX21ppL6xLgu3 5ZJt7jiO9N6vkfSX9o50se23eUks5tKFyaAtA2BaAb5hXSJsjW57TUEsCPN9H4uylJ Sl579IcUgS2sEawqSfQYLm5aXq3OFA8hbNe4VeWKVVwbnYYZGMwnHJuI8dJrg4adU1 8BwZeJMrOJyaQ== From: Eric Biggers To: linux-rdma@vger.kernel.org, Mustafa Ismail , Tatyana Nikolova , Jason Gunthorpe , Leon Romanovsky , Zhu Yanjun , Bernard Metzler Cc: linux-kernel@vger.kernel.org Subject: [PATCH 3/6] RDMA/rxe: switch to using the crc32 library Date: Mon, 27 Jan 2025 14:38:37 -0800 Message-ID: <20250127223840.67280-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250127223840.67280-1-ebiggers@kernel.org> References: <20250127223840.67280-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Eric Biggers Now that the crc32() library function takes advantage of architecture-specific optimizations, it is unnecessary to go through the crypto API. Just use crc32(). This is much simpler, and it improves performance due to eliminating the crypto API overhead. Signed-off-by: Eric Biggers Reviewed-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/Kconfig | 3 +- drivers/infiniband/sw/rxe/rxe.c | 3 -- drivers/infiniband/sw/rxe/rxe.h | 1 - drivers/infiniband/sw/rxe/rxe_icrc.c | 61 ++------------------------- drivers/infiniband/sw/rxe/rxe_loc.h | 1 - drivers/infiniband/sw/rxe/rxe_req.c | 1 - drivers/infiniband/sw/rxe/rxe_verbs.c | 4 -- drivers/infiniband/sw/rxe/rxe_verbs.h | 1 - 8 files changed, 5 insertions(+), 70 deletions(-) diff --git a/drivers/infiniband/sw/rxe/Kconfig b/drivers/infiniband/sw/rxe/Kconfig index 06b8dc5093f77..c180e7ebcfc5b 100644 --- a/drivers/infiniband/sw/rxe/Kconfig +++ b/drivers/infiniband/sw/rxe/Kconfig @@ -2,12 +2,11 @@ config RDMA_RXE tristate "Software RDMA over Ethernet (RoCE) driver" depends on INET && PCI && INFINIBAND depends on INFINIBAND_VIRT_DMA select NET_UDP_TUNNEL - select CRYPTO - select CRYPTO_CRC32 + select CRC32 help This driver implements the InfiniBand RDMA transport over the Linux network stack. It enables a system with a standard Ethernet adapter to interoperate with a RoCE adapter or with another system running the RXE driver. diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index 1ba4a0c8726ae..f8ac79ef70143 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -29,13 +29,10 @@ void rxe_dealloc(struct ib_device *ib_dev) rxe_pool_cleanup(&rxe->mr_pool); rxe_pool_cleanup(&rxe->mw_pool); WARN_ON(!RB_EMPTY_ROOT(&rxe->mcg_tree)); - if (rxe->tfm) - crypto_free_shash(rxe->tfm); - mutex_destroy(&rxe->usdev_lock); } /* initialize rxe device parameters */ static void rxe_init_device_param(struct rxe_dev *rxe) diff --git a/drivers/infiniband/sw/rxe/rxe.h b/drivers/infiniband/sw/rxe/rxe.h index fe7f970667325..8db65731499d0 100644 --- a/drivers/infiniband/sw/rxe/rxe.h +++ b/drivers/infiniband/sw/rxe/rxe.h @@ -19,11 +19,10 @@ #include #include #include #include #include -#include #include "rxe_net.h" #include "rxe_opcode.h" #include "rxe_hdr.h" #include "rxe_param.h" diff --git a/drivers/infiniband/sw/rxe/rxe_icrc.c b/drivers/infiniband/sw/rxe/rxe_icrc.c index c7b0b4673b959..63d03f0f71e38 100644 --- a/drivers/infiniband/sw/rxe/rxe_icrc.c +++ b/drivers/infiniband/sw/rxe/rxe_icrc.c @@ -7,62 +7,10 @@ #include #include "rxe.h" #include "rxe_loc.h" -/** - * rxe_icrc_init() - Initialize crypto function for computing crc32 - * @rxe: rdma_rxe device object - * - * Return: 0 on success else an error - */ -int rxe_icrc_init(struct rxe_dev *rxe) -{ - struct crypto_shash *tfm; - - tfm = crypto_alloc_shash("crc32", 0, 0); - if (IS_ERR(tfm)) { - rxe_dbg_dev(rxe, "failed to init crc32 algorithm err: %ld\n", - PTR_ERR(tfm)); - return PTR_ERR(tfm); - } - - rxe->tfm = tfm; - - return 0; -} - -/** - * rxe_crc32() - Compute cumulative crc32 for a contiguous segment - * @rxe: rdma_rxe device object - * @crc: starting crc32 value from previous segments - * @next: starting address of current segment - * @len: length of current segment - * - * Return: the cumulative crc32 checksum - */ -static u32 rxe_crc32(struct rxe_dev *rxe, u32 crc, void *next, size_t len) -{ - u32 icrc; - int err; - - SHASH_DESC_ON_STACK(shash, rxe->tfm); - - shash->tfm = rxe->tfm; - *(u32 *)shash_desc_ctx(shash) = crc; - err = crypto_shash_update(shash, next, len); - if (unlikely(err)) { - rxe_dbg_dev(rxe, "failed crc calculation, err: %d\n", err); - return crc32_le(crc, next, len); - } - - icrc = *(u32 *)shash_desc_ctx(shash); - barrier_data(shash_desc_ctx(shash)); - - return icrc; -} - /** * rxe_icrc() - Compute the ICRC of a packet * @skb: packet buffer * @pkt: packet information * @@ -117,19 +65,18 @@ static u32 rxe_icrc(struct sk_buff *skb, struct rxe_pkt_info *pkt) /* exclude bth.resv8a */ bth->qpn |= cpu_to_be32(~BTH_QPN_MASK); /* Update the CRC with the first part of the headers. */ - crc = rxe_crc32(pkt->rxe, crc, pshdr, hdr_size + RXE_BTH_BYTES); + crc = crc32(crc, pshdr, hdr_size + RXE_BTH_BYTES); /* Update the CRC with the remainder of the headers. */ - crc = rxe_crc32(pkt->rxe, crc, pkt->hdr + RXE_BTH_BYTES, - rxe_opcode[pkt->opcode].length - RXE_BTH_BYTES); + crc = crc32(crc, pkt->hdr + RXE_BTH_BYTES, + rxe_opcode[pkt->opcode].length - RXE_BTH_BYTES); /* Update the CRC with the payload. */ - crc = rxe_crc32(pkt->rxe, crc, payload_addr(pkt), - payload_size(pkt) + bth_pad(pkt)); + crc = crc32(crc, payload_addr(pkt), payload_size(pkt) + bth_pad(pkt)); /* Invert the CRC and return it. */ return ~crc; } diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h index ded46119151bb..c57ab8975c5d1 100644 --- a/drivers/infiniband/sw/rxe/rxe_loc.h +++ b/drivers/infiniband/sw/rxe/rxe_loc.h @@ -166,11 +166,10 @@ int rxe_completer(struct rxe_qp *qp); int rxe_requester(struct rxe_qp *qp); int rxe_sender(struct rxe_qp *qp); int rxe_receiver(struct rxe_qp *qp); /* rxe_icrc.c */ -int rxe_icrc_init(struct rxe_dev *rxe); int rxe_icrc_check(struct sk_buff *skb, struct rxe_pkt_info *pkt); void rxe_icrc_generate(struct sk_buff *skb, struct rxe_pkt_info *pkt); void rxe_resp_queue_pkt(struct rxe_qp *qp, struct sk_buff *skb); diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 87a02f0deb000..9d0392df8a92f 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -3,11 +3,10 @@ * Copyright (c) 2016 Mellanox Technologies Ltd. All rights reserved. * Copyright (c) 2015 System Fabric Works, Inc. All rights reserved. */ #include -#include #include "rxe.h" #include "rxe_loc.h" #include "rxe_queue.h" diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 6152a0fdfc8ca..c05379f8b4f57 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -1531,14 +1531,10 @@ int rxe_register_device(struct rxe_dev *rxe, const char *ibdev_name, ib_set_device_ops(dev, &rxe_dev_ops); err = ib_device_set_netdev(&rxe->ib_dev, ndev, 1); if (err) return err; - err = rxe_icrc_init(rxe); - if (err) - return err; - err = ib_register_device(dev, ibdev_name, NULL); if (err) rxe_dbg_dev(rxe, "failed with error %d\n", err); /* diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 6573ceec0ef58..6e31134a5fa5b 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -400,11 +400,10 @@ struct rxe_dev { u64 mmap_offset; atomic64_t stats_counters[RXE_NUM_OF_COUNTERS]; struct rxe_port port; - struct crypto_shash *tfm; }; static inline struct net_device *rxe_ib_device_get_netdev(struct ib_device *dev) { return ib_device_get_netdev(dev, RXE_PORT); From patchwork Mon Jan 27 22:38:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13951805 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 18D2F1DD0F6; Mon, 27 Jan 2025 22:38:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738017540; cv=none; b=cvuIy/hUcY1c61AaVBI9bOp3sIkNNuwwSTziAusi/iVu7WqJdGzosIBmwzaNo76hUOlAdEnoxbwuSVPrInpYF6oen6Rr0e5iZJi2TluLJimNXA6EtxBXR/MBcvT5Ghnjk7xxlsK8poSSpCzCkpd1SvpATNWj9ng6Qxd9zaPkS3Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738017540; c=relaxed/simple; bh=wXKtkxdknQtnti8i4l7gtaPVUVhUdaUIoxQBxtfduBo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ohQyXQ8KbO4lOYGqeFel17WJ313V1UQXB5JkKq/wU0AGpywyPU2u1If+1I1XqMu1yCuc3qLzAi2rCRJBRLbB9wPD9in+2B9hTigLAlE9UeLr1NMV6mMOrLsxOMt2JUZDeeRbGUe53PxPYUJO6zs9zwGoKMIzbepaoEFqZwv21pI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pdzbnMPS; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pdzbnMPS" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 700EAC4CEE8; Mon, 27 Jan 2025 22:38:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738017539; bh=wXKtkxdknQtnti8i4l7gtaPVUVhUdaUIoxQBxtfduBo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pdzbnMPSe+PYR7PpBXDlvwhdxdoGHV0yPQRIfr692XN0V4eRrdmC0B0KN+mvLrexD USB5lp1e+95v5w3VLy/RQac+Ic4KL0ngQ4/QyyAEqVV+pNdlWrIC6DsmtOqH/nnfl8 hdfB/sZptCCGEP0pRD9c7+ArkFlcR+bUwYl6BfALyP2WxX6clJLAL2eniY2fTKWmNa qcVzHZOfDcBxyko3Ef59gV4pPpmk1VgQDbv3mBrho/rCECXFluMCsYua80t8YEb17w aSF2l2i4FMNR1obBHXcKU8+4qm+HgNUBt3FcQPLPUYEgv3rpoOr9ITR3HGPsrZlrUt qZj6/cGprsYNg== From: Eric Biggers To: linux-rdma@vger.kernel.org, Mustafa Ismail , Tatyana Nikolova , Jason Gunthorpe , Leon Romanovsky , Zhu Yanjun , Bernard Metzler Cc: linux-kernel@vger.kernel.org Subject: [PATCH 4/6] RDMA/irdma: switch to using the crc32c library Date: Mon, 27 Jan 2025 14:38:38 -0800 Message-ID: <20250127223840.67280-5-ebiggers@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250127223840.67280-1-ebiggers@kernel.org> References: <20250127223840.67280-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Eric Biggers Now that the crc32c() library function directly takes advantage of architecture-specific optimizations, it is unnecessary to go through the crypto API. Just use crc32c(). This is much simpler, and it improves performance due to eliminating the crypto API overhead. Signed-off-by: Eric Biggers --- drivers/infiniband/hw/irdma/Kconfig | 1 + drivers/infiniband/hw/irdma/main.h | 1 - drivers/infiniband/hw/irdma/osdep.h | 6 +--- drivers/infiniband/hw/irdma/puda.c | 19 +++++------- drivers/infiniband/hw/irdma/puda.h | 5 +-- drivers/infiniband/hw/irdma/utils.c | 47 ++--------------------------- 6 files changed, 12 insertions(+), 67 deletions(-) diff --git a/drivers/infiniband/hw/irdma/Kconfig b/drivers/infiniband/hw/irdma/Kconfig index b6f9c41bca51d..5f49a58590ed7 100644 --- a/drivers/infiniband/hw/irdma/Kconfig +++ b/drivers/infiniband/hw/irdma/Kconfig @@ -5,8 +5,9 @@ config INFINIBAND_IRDMA depends on IPV6 || !IPV6 depends on PCI depends on ICE && I40E select GENERIC_ALLOCATOR select AUXILIARY_BUS + select CRC32 help This is an Intel(R) Ethernet Protocol Driver for RDMA driver that support E810 (iWARP/RoCE) and X722 (iWARP) network devices. diff --git a/drivers/infiniband/hw/irdma/main.h b/drivers/infiniband/hw/irdma/main.h index 9f0ed6e844711..0705ef3d72a93 100644 --- a/drivers/infiniband/hw/irdma/main.h +++ b/drivers/infiniband/hw/irdma/main.h @@ -28,11 +28,10 @@ #ifndef CONFIG_64BIT #include #endif #include #include -#include #include #include #include #include #include diff --git a/drivers/infiniband/hw/irdma/osdep.h b/drivers/infiniband/hw/irdma/osdep.h index ddf02a462efa2..4b4f78288d12e 100644 --- a/drivers/infiniband/hw/irdma/osdep.h +++ b/drivers/infiniband/hw/irdma/osdep.h @@ -4,11 +4,10 @@ #define IRDMA_OSDEP_H #include #include #include -#include #include #define STATS_TIMER_DELAY 60000 struct irdma_dma_info { @@ -41,19 +40,16 @@ struct ib_device *to_ibdev(struct irdma_sc_dev *dev); void irdma_ieq_mpa_crc_ae(struct irdma_sc_dev *dev, struct irdma_sc_qp *qp); enum irdma_status_code irdma_vf_wait_vchnl_resp(struct irdma_sc_dev *dev); bool irdma_vf_clear_to_send(struct irdma_sc_dev *dev); void irdma_add_dev_ref(struct irdma_sc_dev *dev); void irdma_put_dev_ref(struct irdma_sc_dev *dev); -int irdma_ieq_check_mpacrc(struct shash_desc *desc, void *addr, u32 len, - u32 val); +int irdma_ieq_check_mpacrc(const void *addr, u32 len, u32 val); struct irdma_sc_qp *irdma_ieq_get_qp(struct irdma_sc_dev *dev, struct irdma_puda_buf *buf); void irdma_send_ieq_ack(struct irdma_sc_qp *qp); void irdma_ieq_update_tcpip_info(struct irdma_puda_buf *buf, u16 len, u32 seqnum); -void irdma_free_hash_desc(struct shash_desc *hash_desc); -int irdma_init_hash_desc(struct shash_desc **hash_desc); int irdma_puda_get_tcpip_info(struct irdma_puda_cmpl_info *info, struct irdma_puda_buf *buf); int irdma_cqp_sds_cmd(struct irdma_sc_dev *dev, struct irdma_update_sds_info *info); int irdma_cqp_manage_hmc_fcn_cmd(struct irdma_sc_dev *dev, diff --git a/drivers/infiniband/hw/irdma/puda.c b/drivers/infiniband/hw/irdma/puda.c index 7e3f9bca2c235..694e5a9ed15d0 100644 --- a/drivers/infiniband/hw/irdma/puda.c +++ b/drivers/infiniband/hw/irdma/puda.c @@ -921,12 +921,10 @@ void irdma_puda_dele_rsrc(struct irdma_sc_vsi *vsi, enum puda_rsrc_type type, return; } switch (rsrc->cmpl) { case PUDA_HASH_CRC_COMPLETE: - irdma_free_hash_desc(rsrc->hash_desc); - fallthrough; case PUDA_QP_CREATED: irdma_qp_rem_qos(&rsrc->qp); if (!reset) irdma_puda_free_qp(rsrc); @@ -1093,19 +1091,16 @@ int irdma_puda_create_rsrc(struct irdma_sc_vsi *vsi, ret = irdma_puda_replenish_rq(rsrc, true); if (ret) goto error; if (info->type == IRDMA_PUDA_RSRC_TYPE_IEQ) { - if (!irdma_init_hash_desc(&rsrc->hash_desc)) { - rsrc->check_crc = true; - rsrc->cmpl = PUDA_HASH_CRC_COMPLETE; - ret = 0; - } + rsrc->check_crc = true; + rsrc->cmpl = PUDA_HASH_CRC_COMPLETE; } irdma_sc_ccq_arm(&rsrc->cq); - return ret; + return 0; error: irdma_puda_dele_rsrc(vsi, info->type, false); return ret; @@ -1394,12 +1389,12 @@ static int irdma_ieq_handle_partial(struct irdma_puda_rsrc *ieq, irdma_ieq_update_tcpip_info(txbuf, fpdu_len, seqnum); crcptr = txbuf->data + fpdu_len - 4; mpacrc = *(u32 *)crcptr; if (ieq->check_crc) { - status = irdma_ieq_check_mpacrc(ieq->hash_desc, txbuf->data, - (fpdu_len - 4), mpacrc); + status = irdma_ieq_check_mpacrc(txbuf->data, fpdu_len - 4, + mpacrc); if (status) { ibdev_dbg(to_ibdev(ieq->dev), "IEQ: error bad crc\n"); goto error; } } @@ -1463,12 +1458,12 @@ static int irdma_ieq_process_buf(struct irdma_puda_rsrc *ieq, break; } crcptr = datap + fpdu_len - 4; mpacrc = *(u32 *)crcptr; if (ieq->check_crc) - ret = irdma_ieq_check_mpacrc(ieq->hash_desc, datap, - fpdu_len - 4, mpacrc); + ret = irdma_ieq_check_mpacrc(datap, fpdu_len - 4, + mpacrc); if (ret) { list_add(&buf->list, rxlist); ibdev_dbg(to_ibdev(ieq->dev), "ERR: IRDMA_ERR_MPA_CRC\n"); return -EINVAL; diff --git a/drivers/infiniband/hw/irdma/puda.h b/drivers/infiniband/hw/irdma/puda.h index bc6d9514c9c10..2fc638f2b1434 100644 --- a/drivers/infiniband/hw/irdma/puda.h +++ b/drivers/infiniband/hw/irdma/puda.h @@ -117,11 +117,10 @@ struct irdma_puda_rsrc { u64 *rq_wrid_array; u32 compl_rxwqe_idx; u32 rx_wqe_idx; u32 rxq_invalid_cnt; u32 tx_wqe_avail_cnt; - struct shash_desc *hash_desc; struct list_head txpend; struct list_head bufpool; /* free buffers pool list for recv and xmit */ u32 alloc_buf_count; u32 avail_buf_count; /* snapshot of currently available buffers */ spinlock_t bufpool_lock; @@ -161,14 +160,12 @@ int irdma_puda_poll_cmpl(struct irdma_sc_dev *dev, struct irdma_sc_cq *cq, struct irdma_sc_qp *irdma_ieq_get_qp(struct irdma_sc_dev *dev, struct irdma_puda_buf *buf); int irdma_puda_get_tcpip_info(struct irdma_puda_cmpl_info *info, struct irdma_puda_buf *buf); -int irdma_ieq_check_mpacrc(struct shash_desc *desc, void *addr, u32 len, u32 val); -int irdma_init_hash_desc(struct shash_desc **desc); +int irdma_ieq_check_mpacrc(const void *addr, u32 len, u32 val); void irdma_ieq_mpa_crc_ae(struct irdma_sc_dev *dev, struct irdma_sc_qp *qp); -void irdma_free_hash_desc(struct shash_desc *desc); void irdma_ieq_update_tcpip_info(struct irdma_puda_buf *buf, u16 len, u32 seqnum); int irdma_cqp_qp_create_cmd(struct irdma_sc_dev *dev, struct irdma_sc_qp *qp); int irdma_cqp_cq_create_cmd(struct irdma_sc_dev *dev, struct irdma_sc_cq *cq); int irdma_cqp_qp_destroy_cmd(struct irdma_sc_dev *dev, struct irdma_sc_qp *qp); void irdma_cqp_cq_destroy_cmd(struct irdma_sc_dev *dev, struct irdma_sc_cq *cq); diff --git a/drivers/infiniband/hw/irdma/utils.c b/drivers/infiniband/hw/irdma/utils.c index 0e594122baa78..57c3335be103c 100644 --- a/drivers/infiniband/hw/irdma/utils.c +++ b/drivers/infiniband/hw/irdma/utils.c @@ -1271,62 +1271,19 @@ void irdma_ieq_mpa_crc_ae(struct irdma_sc_dev *dev, struct irdma_sc_qp *qp) info.ae_code = IRDMA_AE_LLP_RECEIVED_MPA_CRC_ERROR; info.ae_src = IRDMA_AE_SOURCE_RQ; irdma_gen_ae(rf, qp, &info, false); } -/** - * irdma_init_hash_desc - initialize hash for crc calculation - * @desc: cryption type - */ -int irdma_init_hash_desc(struct shash_desc **desc) -{ - struct crypto_shash *tfm; - struct shash_desc *tdesc; - - tfm = crypto_alloc_shash("crc32c", 0, 0); - if (IS_ERR(tfm)) - return -EINVAL; - - tdesc = kzalloc(sizeof(*tdesc) + crypto_shash_descsize(tfm), - GFP_KERNEL); - if (!tdesc) { - crypto_free_shash(tfm); - return -EINVAL; - } - - tdesc->tfm = tfm; - *desc = tdesc; - - return 0; -} - -/** - * irdma_free_hash_desc - free hash desc - * @desc: to be freed - */ -void irdma_free_hash_desc(struct shash_desc *desc) -{ - if (desc) { - crypto_free_shash(desc->tfm); - kfree(desc); - } -} - /** * irdma_ieq_check_mpacrc - check if mpa crc is OK - * @desc: desc for hash * @addr: address of buffer for crc * @len: length of buffer * @val: value to be compared */ -int irdma_ieq_check_mpacrc(struct shash_desc *desc, void *addr, u32 len, - u32 val) +int irdma_ieq_check_mpacrc(const void *addr, u32 len, u32 val) { - u32 crc = 0; - - crypto_shash_digest(desc, addr, len, (u8 *)&crc); - if (crc != val) + if (~crc32c(~0, addr, len) != val) return -EINVAL; return 0; } From patchwork Mon Jan 27 22:38:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13951806 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3D2771DD889; Mon, 27 Jan 2025 22:39:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738017540; cv=none; b=SaaKRdly3pME4xdGmN9AoHFeq2E+WxKftG8aMVfg9hrawiH74UQ2Xg30Uz7Yg+c+UUUbN55cUJ2FsbTZ7J/ncNAk1c03u0+5vLmCdJmC8eMaPXulMS+RFE9VxCVR7fDqI9D0quY0yx8tgVvJyVry/hnRImRXHqslD1Z5Q1Lqp+Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738017540; c=relaxed/simple; bh=b91klK3SDu77umDEst57kH6XXlYgewr0E6J0kuKTk0s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qYZFWXjf4CMAn04go2pYGuE9oWxzznZA0/HdVtiLqP+60FVpI4RJXrNiVGATbUKgu2lE6r33yii6hMS4hQzaNj+WS/jI9wcNpEnkwA2I7TncCwX+x44Lv4oIU3nIQIVydoNTZbJ6sFN3x6Dm/zmRnoUUbX/aNWQhCUcwUpLTSIQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CYSg7QUI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CYSg7QUI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CE741C4CEEA; Mon, 27 Jan 2025 22:38:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738017540; bh=b91klK3SDu77umDEst57kH6XXlYgewr0E6J0kuKTk0s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CYSg7QUIbRisEBkWvEqE8+BjW44ZKvV685FKZMox4q/vqJXnb4DGC8CptA3TBipB/ Imx5CeES0q0vtQwNIDMtxJ3FSAS5Up46efvzofdpn2yQf5ZtbTFqfEiwfoa5d/qqrL KrVKpiJ0EMCDc6VixUkXwBg0pglUu3RRbAbn4CwKGtKdkGKGcWzkRBYObzN70y0HHx TzxzfkzagHR5oR/mrbnMJyD399pUUS/0NV/b/CP5/wLVs2gH3xafnQjcb5u51S6F3h xaVfoOPQ0FAao0bwR2DdFDJf472R5R8rWbwFlAs0QVTdNFOYfN/YDbv0PpSTGj5+vS MLkFkiHTGsG+A== From: Eric Biggers To: linux-rdma@vger.kernel.org, Mustafa Ismail , Tatyana Nikolova , Jason Gunthorpe , Leon Romanovsky , Zhu Yanjun , Bernard Metzler Cc: linux-kernel@vger.kernel.org Subject: [PATCH 5/6] RDMA/siw: fix type of CRC field Date: Mon, 27 Jan 2025 14:38:39 -0800 Message-ID: <20250127223840.67280-6-ebiggers@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250127223840.67280-1-ebiggers@kernel.org> References: <20250127223840.67280-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Eric Biggers The usual big endian convention of InfiniBand does not apply to the MPA CRC field, whose transmission is specified in terms of the CRC32 polynomial coefficients. The coefficients are transmitted in descending order from the x^31 coefficient to the x^0 one. When the result is interpreted as a 32-bit integer using the required reverse mapping between bits and polynomial coefficients, it's a __le32. Fix the code to use the correct type. The endianness conversion to little endian was actually already done in crypto_shash_final(). Therefore this patch does not change any behavior, except for adding the missing le32_to_cpu() to the pr_warn() message in siw_get_trailer() which makes the correct values be printed on big endian systems. Signed-off-by: Eric Biggers --- drivers/infiniband/sw/siw/iwarp.h | 2 +- drivers/infiniband/sw/siw/siw.h | 8 ++++---- drivers/infiniband/sw/siw/siw_qp.c | 2 +- drivers/infiniband/sw/siw/siw_qp_rx.c | 16 +++++++++++----- drivers/infiniband/sw/siw/siw_qp_tx.c | 2 +- 5 files changed, 18 insertions(+), 12 deletions(-) diff --git a/drivers/infiniband/sw/siw/iwarp.h b/drivers/infiniband/sw/siw/iwarp.h index 8cf69309827d6..afebb5d8643e3 100644 --- a/drivers/infiniband/sw/siw/iwarp.h +++ b/drivers/infiniband/sw/siw/iwarp.h @@ -79,11 +79,11 @@ struct mpa_marker { /* * maximum MPA trailer */ struct mpa_trailer { __u8 pad[4]; - __be32 crc; + __le32 crc; }; #define MPA_HDR_SIZE 2 #define MPA_CRC_SIZE 4 diff --git a/drivers/infiniband/sw/siw/siw.h b/drivers/infiniband/sw/siw/siw.h index ea5eee50dc39d..50649971f6254 100644 --- a/drivers/infiniband/sw/siw/siw.h +++ b/drivers/infiniband/sw/siw/siw.h @@ -336,26 +336,26 @@ struct siw_rx_fpdu { * Shorthands for short packets w/o payload * to be transmitted more efficient. */ struct siw_send_pkt { struct iwarp_send send; - __be32 crc; + __le32 crc; }; struct siw_write_pkt { struct iwarp_rdma_write write; - __be32 crc; + __le32 crc; }; struct siw_rreq_pkt { struct iwarp_rdma_rreq rreq; - __be32 crc; + __le32 crc; }; struct siw_rresp_pkt { struct iwarp_rdma_rresp rresp; - __be32 crc; + __le32 crc; }; struct siw_iwarp_tx { union { union iwarp_hdr hdr; diff --git a/drivers/infiniband/sw/siw/siw_qp.c b/drivers/infiniband/sw/siw/siw_qp.c index da92cfa2073d7..ea7d2f5c8b8ee 100644 --- a/drivers/infiniband/sw/siw/siw_qp.c +++ b/drivers/infiniband/sw/siw/siw_qp.c @@ -394,11 +394,11 @@ void siw_send_terminate(struct siw_qp *qp) struct iwarp_terminate *term = NULL; union iwarp_hdr *err_hdr = NULL; struct socket *s = qp->attrs.sk; struct siw_rx_stream *srx = &qp->rx_stream; union iwarp_hdr *rx_hdr = &srx->hdr; - u32 crc = 0; + __le32 crc = 0; int num_frags, len_terminate, rv; if (!qp->term_info.valid) return; diff --git a/drivers/infiniband/sw/siw/siw_qp_rx.c b/drivers/infiniband/sw/siw/siw_qp_rx.c index ed4fc39718b49..098e32fb36fb1 100644 --- a/drivers/infiniband/sw/siw/siw_qp_rx.c +++ b/drivers/infiniband/sw/siw/siw_qp_rx.c @@ -951,11 +951,11 @@ int siw_proc_terminate(struct siw_qp *qp) static int siw_get_trailer(struct siw_qp *qp, struct siw_rx_stream *srx) { struct sk_buff *skb = srx->skb; int avail = min(srx->skb_new, srx->fpdu_part_rem); u8 *tbuf = (u8 *)&srx->trailer.crc - srx->pad; - __wsum crc_in, crc_own = 0; + __le32 crc_in, crc_own = 0; siw_dbg_qp(qp, "expected %d, available %d, pad %u\n", srx->fpdu_part_rem, srx->skb_new, srx->pad); skb_copy_bits(skb, srx->skb_offset, tbuf, avail); @@ -969,20 +969,26 @@ static int siw_get_trailer(struct siw_qp *qp, struct siw_rx_stream *srx) if (!srx->mpa_crc_hd) return 0; if (srx->pad) crypto_shash_update(srx->mpa_crc_hd, tbuf, srx->pad); + /* - * CRC32 is computed, transmitted and received directly in NBO, - * so there's never a reason to convert byte order. + * The usual big endian convention of InfiniBand does not apply to the + * CRC field, whose transmission is specified in terms of the CRC32 + * polynomial coefficients. The coefficients are transmitted in + * descending order from the x^31 coefficient to the x^0 one. When the + * result is interpreted as a 32-bit integer using the required reverse + * mapping between bits and polynomial coefficients, it's a __le32. */ crypto_shash_final(srx->mpa_crc_hd, (u8 *)&crc_own); - crc_in = (__force __wsum)srx->trailer.crc; + crc_in = srx->trailer.crc; if (unlikely(crc_in != crc_own)) { pr_warn("siw: crc error. in: %08x, own %08x, op %u\n", - crc_in, crc_own, qp->rx_stream.rdmap_op); + le32_to_cpu(crc_in), le32_to_cpu(crc_own), + qp->rx_stream.rdmap_op); siw_init_terminate(qp, TERM_ERROR_LAYER_LLP, LLP_ETYPE_MPA, LLP_ECODE_RECEIVED_CRC, 0); return -EINVAL; diff --git a/drivers/infiniband/sw/siw/siw_qp_tx.c b/drivers/infiniband/sw/siw/siw_qp_tx.c index a034264c56698..f9db69a82cdd5 100644 --- a/drivers/infiniband/sw/siw/siw_qp_tx.c +++ b/drivers/infiniband/sw/siw/siw_qp_tx.c @@ -242,11 +242,11 @@ static int siw_qp_prepare_tx(struct siw_iwarp_tx *c_tx) else c_tx->pkt.c_tagged.ddp_to = cpu_to_be64(wqe->sqe.raddr); } - *(u32 *)crc = 0; + *(__le32 *)crc = 0; /* * Do complete CRC if enabled and short packet */ if (c_tx->mpa_crc_hd && crypto_shash_digest(c_tx->mpa_crc_hd, (u8 *)&c_tx->pkt, From patchwork Mon Jan 27 22:38:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 13951807 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9BE6A1DDC01; Mon, 27 Jan 2025 22:39:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738017540; cv=none; b=ZmbNxej512NKu3Mb79lBUgcROLEydok8Yfg1SD/kgmhmmjgzBZvQlddLZG7RAdYWaMPUnV5X9pzrY3btX3nTxH+Pn6C1nyGLnki4XVVJgak2rmUUBgCVSyPYo25Iw6FW4xvwoaXfOEx/fir9NLwNnKu/L4nu3GDyo9IIKEcE+yw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738017540; c=relaxed/simple; bh=xFJJ94k5+V4jUZQcU8VYMkRnyDUpNRZJUa8vybUr9tQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Gocw9ZNsAgyv6v8vjiyhUQs0l81hNiDi0B98RlK8ggDT7MGuCBVh8tdQ21xrgq7QuRkEeoRk0nlqQRFVM3m40D9JJ80tf502Q7CE6R+wreZdYb8T2KOjPCfq66M8+lpI64+Iyad60XMdUN6Mem/g9zknApgBfnh6qJsd3PK1W2A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PDVZAc2z; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PDVZAc2z" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3D143C4CEE7; Mon, 27 Jan 2025 22:39:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738017540; bh=xFJJ94k5+V4jUZQcU8VYMkRnyDUpNRZJUa8vybUr9tQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PDVZAc2z5sBt+nUyIGvkev29PEDBNOeuTJzjSBx+7dvKvKuoLHCc/fweeUrPMsdln 6Djgtr/HudZWB6iQVVxyAocth4UZ/xCEyP6fp3MBSHV5Pvv3VJVGmhLUhwuZJNRytP bfwzlPEn8oHK99OgiBIYiTNzFgVJ1LYwZXkuOPx2Nm9Dk1VuPjkHLWa+uwKdX8D52B THcotceHZfBGneSKoG5IuudMkAiZaStzyV382Nh6ydOEcufjEUMxj7RbqSYLLkHoFN xs/pw6ZRMokIW94XNvr/Pzjs3yKUs+WNbzwptbC9vVSMaMFLixRU0oBdgdxH9Cizvm i3xndB3RxrSPA== From: Eric Biggers To: linux-rdma@vger.kernel.org, Mustafa Ismail , Tatyana Nikolova , Jason Gunthorpe , Leon Romanovsky , Zhu Yanjun , Bernard Metzler Cc: linux-kernel@vger.kernel.org Subject: [PATCH 6/6] RDMA/siw: switch to using the crc32c library Date: Mon, 27 Jan 2025 14:38:40 -0800 Message-ID: <20250127223840.67280-7-ebiggers@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250127223840.67280-1-ebiggers@kernel.org> References: <20250127223840.67280-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Eric Biggers Now that the crc32c() library function directly takes advantage of architecture-specific optimizations, it is unnecessary to go through the crypto API. Just use crc32c(). This is much simpler, and it improves performance due to eliminating the crypto API overhead. Signed-off-by: Eric Biggers --- drivers/infiniband/sw/siw/Kconfig | 4 +- drivers/infiniband/sw/siw/siw.h | 38 +++++++++++++++---- drivers/infiniband/sw/siw/siw_main.c | 22 +---------- drivers/infiniband/sw/siw/siw_qp.c | 54 ++++++--------------------- drivers/infiniband/sw/siw/siw_qp_rx.c | 28 ++++++-------- drivers/infiniband/sw/siw/siw_qp_tx.c | 45 ++++++++++------------ drivers/infiniband/sw/siw/siw_verbs.c | 3 -- 7 files changed, 75 insertions(+), 119 deletions(-) diff --git a/drivers/infiniband/sw/siw/Kconfig b/drivers/infiniband/sw/siw/Kconfig index 81b70a3eeb878..ae4a953e2a039 100644 --- a/drivers/infiniband/sw/siw/Kconfig +++ b/drivers/infiniband/sw/siw/Kconfig @@ -1,12 +1,10 @@ config RDMA_SIW tristate "Software RDMA over TCP/IP (iWARP) driver" depends on INET && INFINIBAND depends on INFINIBAND_VIRT_DMA - select LIBCRC32C - select CRYPTO - select CRYPTO_CRC32C + select CRC32 help This driver implements the iWARP RDMA transport over the Linux TCP/IP network stack. It enables a system with a standard Ethernet adapter to interoperate with a iWARP adapter or with another system running the SIW driver. diff --git a/drivers/infiniband/sw/siw/siw.h b/drivers/infiniband/sw/siw/siw.h index 50649971f6254..10c69388b4028 100644 --- a/drivers/infiniband/sw/siw/siw.h +++ b/drivers/infiniband/sw/siw/siw.h @@ -8,11 +8,10 @@ #include #include #include #include -#include #include #include #include #include "iwarp.h" @@ -287,11 +286,12 @@ struct siw_rx_stream { enum siw_rx_state state; union iwarp_hdr hdr; struct mpa_trailer trailer; - struct shash_desc *mpa_crc_hd; + u32 mpa_crc; + u8 mpa_crc_enabled : 1; /* * For each FPDU, main RX loop runs through 3 stages: * Receiving protocol headers, placing DDP payload and receiving * trailer information (CRC + possibly padding). @@ -388,11 +388,12 @@ struct siw_iwarp_tx { u16 ctrl_len; /* ddp+rdmap hdr */ u16 ctrl_sent; int burst; int bytes_unsent; /* ddp payload bytes */ - struct shash_desc *mpa_crc_hd; + u32 mpa_crc; + u8 mpa_crc_enabled : 1; u8 do_crc : 1; /* do crc for segment */ u8 use_sendpage : 1; /* send w/o copy */ u8 tx_suspend : 1; /* stop sending DDP segs. */ u8 pad : 2; /* # pad in current fpdu */ @@ -494,11 +495,10 @@ extern const bool mpa_crc_strict; extern const bool siw_tcp_nagle; extern u_char mpa_version; extern const bool peer_to_peer; extern struct task_struct *siw_tx_thread[]; -extern struct crypto_shash *siw_crypto_shash; extern struct iwarp_msg_info iwarp_pktinfo[RDMAP_TERMINATE + 1]; /* QP general functions */ int siw_qp_modify(struct siw_qp *qp, struct siw_qp_attrs *attr, enum siw_qp_attr_mask mask); @@ -666,10 +666,34 @@ static inline struct siw_sqe *irq_alloc_free(struct siw_qp *qp) return irq_e; } return NULL; } +static inline void siw_crc_init(u32 *crc) +{ + *crc = ~0; +} + +static inline void siw_crc_update(u32 *crc, const void *addr, unsigned int len) +{ + *crc = crc32c(*crc, addr, len); +} + +static inline __le32 siw_crc_final(u32 *crc) +{ + return cpu_to_le32(~*crc); +} + +static inline __le32 siw_crc_oneshot(const void *addr, unsigned int len) +{ + u32 crc; + + siw_crc_init(&crc); + siw_crc_update(&crc, addr, len); + return siw_crc_final(&crc); +} + static inline __wsum siw_csum_update(const void *buff, int len, __wsum sum) { return (__force __wsum)crc32c((__force __u32)sum, buff, len); } @@ -684,15 +708,13 @@ static inline void siw_crc_skb(struct siw_rx_stream *srx, unsigned int len) { const struct skb_checksum_ops siw_cs_ops = { .update = siw_csum_update, .combine = siw_csum_combine, }; - __wsum crc = *(u32 *)shash_desc_ctx(srx->mpa_crc_hd); - crc = __skb_checksum(srx->skb, srx->skb_offset, len, crc, - &siw_cs_ops); - *(u32 *)shash_desc_ctx(srx->mpa_crc_hd) = crc; + srx->mpa_crc = __skb_checksum(srx->skb, srx->skb_offset, len, + srx->mpa_crc, &siw_cs_ops); } #define siw_dbg(ibdev, fmt, ...) \ ibdev_dbg(ibdev, "%s: " fmt, __func__, ##__VA_ARGS__) diff --git a/drivers/infiniband/sw/siw/siw_main.c b/drivers/infiniband/sw/siw/siw_main.c index b17752bd1ecc1..5168307229a9e 100644 --- a/drivers/infiniband/sw/siw/siw_main.c +++ b/drivers/infiniband/sw/siw/siw_main.c @@ -57,11 +57,10 @@ u_char mpa_version = MPA_REVISION_2; * setup, if true. */ const bool peer_to_peer; struct task_struct *siw_tx_thread[NR_CPUS]; -struct crypto_shash *siw_crypto_shash; static int siw_device_register(struct siw_device *sdev, const char *name) { struct ib_device *base_dev = &sdev->base_dev; static int dev_id = 1; @@ -465,24 +464,11 @@ static __init int siw_init_module(void) if (!siw_create_tx_threads()) { pr_info("siw: Could not start any TX thread\n"); rv = -ENOMEM; goto out_error; } - /* - * Locate CRC32 algorithm. If unsuccessful, fail - * loading siw only, if CRC is required. - */ - siw_crypto_shash = crypto_alloc_shash("crc32c", 0, 0); - if (IS_ERR(siw_crypto_shash)) { - pr_info("siw: Loading CRC32c failed: %ld\n", - PTR_ERR(siw_crypto_shash)); - siw_crypto_shash = NULL; - if (mpa_crc_required) { - rv = -EOPNOTSUPP; - goto out_error; - } - } + rv = register_netdevice_notifier(&siw_netdev_nb); if (rv) goto out_error; rdma_link_register(&siw_link_ops); @@ -491,13 +477,10 @@ static __init int siw_init_module(void) return 0; out_error: siw_stop_tx_threads(); - if (siw_crypto_shash) - crypto_free_shash(siw_crypto_shash); - pr_info("SoftIWARP attach failed. Error: %d\n", rv); siw_cm_exit(); siw_destroy_cpulist(siw_cpu_info.num_nodes); @@ -514,13 +497,10 @@ static void __exit siw_exit_module(void) siw_cm_exit(); siw_destroy_cpulist(siw_cpu_info.num_nodes); - if (siw_crypto_shash) - crypto_free_shash(siw_crypto_shash); - pr_info("SoftiWARP detached\n"); } module_init(siw_init_module); module_exit(siw_exit_module); diff --git a/drivers/infiniband/sw/siw/siw_qp.c b/drivers/infiniband/sw/siw/siw_qp.c index ea7d2f5c8b8ee..75c8ca9559391 100644 --- a/drivers/infiniband/sw/siw/siw_qp.c +++ b/drivers/infiniband/sw/siw/siw_qp.c @@ -224,37 +224,10 @@ static int siw_qp_readq_init(struct siw_qp *qp, int irq_size, int orq_size) qp->attrs.orq_size = orq_size; siw_dbg_qp(qp, "ORD %d, IRD %d\n", orq_size, irq_size); return 0; } -static int siw_qp_enable_crc(struct siw_qp *qp) -{ - struct siw_rx_stream *c_rx = &qp->rx_stream; - struct siw_iwarp_tx *c_tx = &qp->tx_ctx; - int size; - - if (siw_crypto_shash == NULL) - return -ENOENT; - - size = crypto_shash_descsize(siw_crypto_shash) + - sizeof(struct shash_desc); - - c_tx->mpa_crc_hd = kzalloc(size, GFP_KERNEL); - c_rx->mpa_crc_hd = kzalloc(size, GFP_KERNEL); - if (!c_tx->mpa_crc_hd || !c_rx->mpa_crc_hd) { - kfree(c_tx->mpa_crc_hd); - kfree(c_rx->mpa_crc_hd); - c_tx->mpa_crc_hd = NULL; - c_rx->mpa_crc_hd = NULL; - return -ENOMEM; - } - c_tx->mpa_crc_hd->tfm = siw_crypto_shash; - c_rx->mpa_crc_hd->tfm = siw_crypto_shash; - - return 0; -} - /* * Send a non signalled READ or WRITE to peer side as negotiated * with MPAv2 P2P setup protocol. The work request is only created * as a current active WR and does not consume Send Queue space. * @@ -581,32 +554,26 @@ void siw_send_terminate(struct siw_qp *qp) rx_hdr->ctrl.mpa_len = cpu_to_be16(real_ddp_len); } term->ctrl.mpa_len = cpu_to_be16(len_terminate - (MPA_HDR_SIZE + MPA_CRC_SIZE)); - if (qp->tx_ctx.mpa_crc_hd) { - crypto_shash_init(qp->tx_ctx.mpa_crc_hd); - if (crypto_shash_update(qp->tx_ctx.mpa_crc_hd, - (u8 *)iov[0].iov_base, - iov[0].iov_len)) - goto out; - + if (qp->tx_ctx.mpa_crc_enabled) { + siw_crc_init(&qp->tx_ctx.mpa_crc); + siw_crc_update(&qp->tx_ctx.mpa_crc, + iov[0].iov_base, iov[0].iov_len); if (num_frags == 3) { - if (crypto_shash_update(qp->tx_ctx.mpa_crc_hd, - (u8 *)iov[1].iov_base, - iov[1].iov_len)) - goto out; + siw_crc_update(&qp->tx_ctx.mpa_crc, + iov[1].iov_base, iov[1].iov_len); } - crypto_shash_final(qp->tx_ctx.mpa_crc_hd, (u8 *)&crc); + crc = siw_crc_final(&qp->tx_ctx.mpa_crc); } rv = kernel_sendmsg(s, &msg, iov, num_frags, len_terminate); siw_dbg_qp(qp, "sent TERM: %s, layer %d, type %d, code %d (%d bytes)\n", rv == len_terminate ? "success" : "failure", __rdmap_term_layer(term), __rdmap_term_etype(term), __rdmap_term_ecode(term), rv); -out: kfree(term); kfree(err_hdr); } /* @@ -641,13 +608,14 @@ static int siw_qp_nextstate_from_idle(struct siw_qp *qp, int rv = 0; switch (attrs->state) { case SIW_QP_STATE_RTS: if (attrs->flags & SIW_MPA_CRC) { - rv = siw_qp_enable_crc(qp); - if (rv) - break; + siw_crc_init(&qp->tx_ctx.mpa_crc); + qp->tx_ctx.mpa_crc_enabled = 1; + siw_crc_init(&qp->rx_stream.mpa_crc); + qp->rx_stream.mpa_crc_enabled = 1; } if (!(mask & SIW_QP_ATTR_LLP_HANDLE)) { siw_dbg_qp(qp, "no socket\n"); rv = -EINVAL; break; diff --git a/drivers/infiniband/sw/siw/siw_qp_rx.c b/drivers/infiniband/sw/siw/siw_qp_rx.c index 098e32fb36fb1..2765bd0df2d0c 100644 --- a/drivers/infiniband/sw/siw/siw_qp_rx.c +++ b/drivers/infiniband/sw/siw/siw_qp_rx.c @@ -65,14 +65,14 @@ static int siw_rx_umem(struct siw_rx_stream *srx, struct siw_umem *umem, pr_warn("siw: [QP %u]: %s, len %d, page %p, rv %d\n", qp_id(rx_qp(srx)), __func__, len, p, rv); return -EFAULT; } - if (srx->mpa_crc_hd) { + if (srx->mpa_crc_enabled) { if (rdma_is_kernel_res(&rx_qp(srx)->base_qp.res)) { - crypto_shash_update(srx->mpa_crc_hd, - (u8 *)(dest + pg_off), bytes); + siw_crc_update(&srx->mpa_crc, dest + pg_off, + bytes); kunmap_atomic(dest); } else { kunmap_atomic(dest); /* * Do CRC on original, not target buffer. @@ -112,12 +112,12 @@ static int siw_rx_kva(struct siw_rx_stream *srx, void *kva, int len) pr_warn("siw: [QP %u]: %s, len %d, kva 0x%pK, rv %d\n", qp_id(rx_qp(srx)), __func__, len, kva, rv); return rv; } - if (srx->mpa_crc_hd) - crypto_shash_update(srx->mpa_crc_hd, (u8 *)kva, len); + if (srx->mpa_crc_enabled) + siw_crc_update(&srx->mpa_crc, kva, len); srx->skb_offset += len; srx->skb_copied += len; srx->skb_new -= len; @@ -964,25 +964,24 @@ static int siw_get_trailer(struct siw_qp *qp, struct siw_rx_stream *srx) srx->fpdu_part_rem -= avail; if (srx->fpdu_part_rem) return -EAGAIN; - if (!srx->mpa_crc_hd) + if (!srx->mpa_crc_enabled) return 0; if (srx->pad) - crypto_shash_update(srx->mpa_crc_hd, tbuf, srx->pad); - + siw_crc_update(&srx->mpa_crc, tbuf, srx->pad); /* * The usual big endian convention of InfiniBand does not apply to the * CRC field, whose transmission is specified in terms of the CRC32 * polynomial coefficients. The coefficients are transmitted in * descending order from the x^31 coefficient to the x^0 one. When the * result is interpreted as a 32-bit integer using the required reverse * mapping between bits and polynomial coefficients, it's a __le32. */ - crypto_shash_final(srx->mpa_crc_hd, (u8 *)&crc_own); + crc_own = siw_crc_final(&srx->mpa_crc); crc_in = srx->trailer.crc; if (unlikely(crc_in != crc_own)) { pr_warn("siw: crc error. in: %08x, own %08x, op %u\n", le32_to_cpu(crc_in), le32_to_cpu(crc_own), @@ -1097,17 +1096,14 @@ static int siw_get_hdr(struct siw_rx_stream *srx) * the current tagged or untagged message gets eventually completed * w/o intersection from another message of the same type * (tagged/untagged). E.g., a WRITE can get intersected by a SEND, * but not by a READ RESPONSE etc. */ - if (srx->mpa_crc_hd) { - /* - * Restart CRC computation - */ - crypto_shash_init(srx->mpa_crc_hd); - crypto_shash_update(srx->mpa_crc_hd, (u8 *)c_hdr, - srx->fpdu_part_rcvd); + if (srx->mpa_crc_enabled) { + /* Restart CRC computation */ + siw_crc_init(&srx->mpa_crc); + siw_crc_update(&srx->mpa_crc, c_hdr, srx->fpdu_part_rcvd); } if (frx->more_ddp_segs) { frx->first_ddp_seg = 0; if (frx->prev_rdmap_op != opcode) { pr_warn("siw: packet intersection: %u : %u\n", diff --git a/drivers/infiniband/sw/siw/siw_qp_tx.c b/drivers/infiniband/sw/siw/siw_qp_tx.c index f9db69a82cdd5..ff6951fa64bc3 100644 --- a/drivers/infiniband/sw/siw/siw_qp_tx.c +++ b/drivers/infiniband/sw/siw/siw_qp_tx.c @@ -246,14 +246,13 @@ static int siw_qp_prepare_tx(struct siw_iwarp_tx *c_tx) *(__le32 *)crc = 0; /* * Do complete CRC if enabled and short packet */ - if (c_tx->mpa_crc_hd && - crypto_shash_digest(c_tx->mpa_crc_hd, (u8 *)&c_tx->pkt, - c_tx->ctrl_len, (u8 *)crc) != 0) - return -EINVAL; + if (c_tx->mpa_crc_enabled) + *(__le32 *)crc = siw_crc_oneshot(&c_tx->pkt, + c_tx->ctrl_len); c_tx->ctrl_len += MPA_CRC_SIZE; return PKT_COMPLETE; } c_tx->ctrl_len += MPA_CRC_SIZE; @@ -480,13 +479,12 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s) iov[seg].iov_base = ib_virt_dma_to_ptr(sge->laddr + sge_off); iov[seg].iov_len = sge_len; if (do_crc) - crypto_shash_update(c_tx->mpa_crc_hd, - iov[seg].iov_base, - sge_len); + siw_crc_update(&c_tx->mpa_crc, + iov[seg].iov_base, sge_len); sge_off += sge_len; data_len -= sge_len; seg++; goto sge_done; } @@ -514,19 +512,18 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s) kmap_mask |= BIT(seg); iov[seg].iov_base = kaddr + fp_off; iov[seg].iov_len = plen; if (do_crc) - crypto_shash_update( - c_tx->mpa_crc_hd, + siw_crc_update( + &c_tx->mpa_crc, iov[seg].iov_base, plen); } else if (do_crc) { kaddr = kmap_local_page(p); - crypto_shash_update(c_tx->mpa_crc_hd, - kaddr + fp_off, - plen); + siw_crc_update(&c_tx->mpa_crc, + kaddr + fp_off, plen); kunmap_local(kaddr); } } else { /* * Cast to an uintptr_t to preserve all 64 bits @@ -534,14 +531,13 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s) */ u64 va = sge->laddr + sge_off; page_array[seg] = ib_virt_dma_to_page(va); if (do_crc) - crypto_shash_update( - c_tx->mpa_crc_hd, - ib_virt_dma_to_ptr(va), - plen); + siw_crc_update(&c_tx->mpa_crc, + ib_virt_dma_to_ptr(va), + plen); } sge_len -= plen; sge_off += plen; data_len -= plen; @@ -574,18 +570,18 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s) } if (c_tx->pad) { *(u32 *)c_tx->trailer.pad = 0; if (do_crc) - crypto_shash_update(c_tx->mpa_crc_hd, - (u8 *)&c_tx->trailer.crc - c_tx->pad, - c_tx->pad); + siw_crc_update(&c_tx->mpa_crc, + (u8 *)&c_tx->trailer.crc - c_tx->pad, + c_tx->pad); } - if (!c_tx->mpa_crc_hd) + if (!c_tx->mpa_crc_enabled) c_tx->trailer.crc = 0; else if (do_crc) - crypto_shash_final(c_tx->mpa_crc_hd, (u8 *)&c_tx->trailer.crc); + c_tx->trailer.crc = siw_crc_final(&c_tx->mpa_crc); data_len = c_tx->bytes_unsent; if (c_tx->use_sendpage) { rv = siw_0copy_tx(s, page_array, &wqe->sqe.sge[c_tx->sge_idx], @@ -734,14 +730,13 @@ static void siw_prepare_fpdu(struct siw_qp *qp, struct siw_wqe *wqe) htons(c_tx->ctrl_len + data_len - MPA_HDR_SIZE); /* * Init MPA CRC computation */ - if (c_tx->mpa_crc_hd) { - crypto_shash_init(c_tx->mpa_crc_hd); - crypto_shash_update(c_tx->mpa_crc_hd, (u8 *)&c_tx->pkt, - c_tx->ctrl_len); + if (c_tx->mpa_crc_enabled) { + siw_crc_init(&c_tx->mpa_crc); + siw_crc_update(&c_tx->mpa_crc, &c_tx->pkt, c_tx->ctrl_len); c_tx->do_crc = 1; } } /* diff --git a/drivers/infiniband/sw/siw/siw_verbs.c b/drivers/infiniband/sw/siw/siw_verbs.c index 5ac8bd450d240..fd7b266a221b2 100644 --- a/drivers/infiniband/sw/siw/siw_verbs.c +++ b/drivers/infiniband/sw/siw/siw_verbs.c @@ -629,13 +629,10 @@ int siw_destroy_qp(struct ib_qp *base_qp, struct ib_udata *udata) siw_cep_put(qp->cep); qp->cep = NULL; } up_write(&qp->state_lock); - kfree(qp->tx_ctx.mpa_crc_hd); - kfree(qp->rx_stream.mpa_crc_hd); - qp->scq = qp->rcq = NULL; siw_qp_put(qp); wait_for_completion(&qp->qp_free);