From patchwork Tue Feb 20 06:44:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13563462 X-Patchwork-Delegate: herbert@gondor.apana.org.au Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 207F25A4C4; Tue, 20 Feb 2024 06:44:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708411481; cv=none; b=WMrNtITXgdaoa1WmXM5w4a4o2OeoGY50Q+DaMFXi1+zBIyU701FnUg31B/EFvH0RKMaOFL5PV2UFMZF95nvKEQ9QHnPV5Dlv6tz85pPe/OufcHem1JpsLN5/mqs4I7rekYEqht7EaNSz3qsqSrTKtDLAFJKhwOwwdU37bauUYOE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708411481; c=relaxed/simple; bh=oQim727lZINnY49Gi9z7V9PnX8PQ7vzU9hPZtVyzet8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Wce9l24sqXCxy9CNG4g/CPjbWzg0xxy9wM6BPNxKjc8PVkN7kaLjJj5oHPRAFBbQottWVoNkxEzbd9gY4R11TeYdLNV5UjalS3JjbBDm2lUT9Yv6slDeYOgLjl7vmvEhA8mJJY08luNYSPp/Yo1dXyIno/1LB8IIIYQuOSnjYPs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KcNJbOon; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KcNJbOon" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1d932f6ccfaso41332415ad.1; Mon, 19 Feb 2024 22:44:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708411479; x=1709016279; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VwTIFpBnJrjGyIT/20HYU3RPVaQg5c8zhvFCBIT6Ljs=; b=KcNJbOonnPzzRUBBbfLkj0GMcgkGITFQhJ0yg8uDtpRsOz7u2J1L3GJdQrirUT7yj+ X4ZCmBhsX7y+silEIF08GUIG8RPW/dykcSFTD4viRE+GadJssDoV/AGygVxI71k3p3QY o6DkbsB8KVxmXRikGFrQsAh16dEOs8Ayr7bASp4mKstd/qq0mde2XOhpHkzTGqVHxr3S nWHY8+b2Ta1ngEjLkzxXxqOs5cTTlE2yl6M/DAS5zuc/uveEVCsjeHgX7kLqn82+f9+U ptki9uUWi6itIa8k2ucgmEqPIB5LWpDUql2kTHFUuW1dJMEEpxW8DIkY0FcnjEzT/fbj UR9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708411479; x=1709016279; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VwTIFpBnJrjGyIT/20HYU3RPVaQg5c8zhvFCBIT6Ljs=; b=QVr7bSGzOUszWkvWJyoRWf6GFSCQlO7a0c7W77s6on2DrYiSVBc39Cq7ZFmBjJp2RC j2+0dhObLWuQ/m9L7UbxByinMXPk3NmocK/WSc06t5XXEYwg/y+C19kT1HhtdQxQ3X7j iBZEuehhld1hiol8W7Ribk1TPvxEQwZIg3GssXLJ89ctUwJf2MiMXI1C6J8/lg/Lanbl gvjDgQo0VZVohzO2pClMmPpPvjvYpC1o2ZwIYUORiWHN9uifhM5+LIBD1W92nXdZhejQ Y58f/vOKzv3hlyfJAMSfu2v/GUfrpje7Tsbzzg7F3Nsv8tYCiryVXI1M9Mt/eZoHaE2C L1ag== X-Forwarded-Encrypted: i=1; AJvYcCU449GkHUTNKEuJwyuCvWMVoEWZFZLrwKT0uUpCJG8wkOIT4VdAMSpXnIRilaO66vxw9phriOExEWKhPLkSlMHVKz3uKZ6wl35iQHqQATWMR0omdiTsWb3gWswu1/YGd8t632nKWq6slDPa X-Gm-Message-State: AOJu0Yz0+Zp/RjD9IFSxwf/ds0ingkEYQRi4oBLvy27xB4AmNCn8TclA 94Sjv1vUxUHi8HOFWuUiz7BLSaVi5Vp2coHgP6WcBGFLNVXr1L// X-Google-Smtp-Source: AGHT+IGai50/e9Dq4GggyO4YXfBocAVgdbPmMRi2mn4PPi0R3OTzklhwKOf6W7+VfocVSf50DBd0sg== X-Received: by 2002:a17:902:dac8:b0:1db:cea0:9d62 with SMTP id q8-20020a170902dac800b001dbcea09d62mr7790208plx.9.1708411479355; Mon, 19 Feb 2024 22:44:39 -0800 (PST) Received: from barry-desktop.hub ([2407:7000:8942:5500:a5bd:9c11:af2b:aecf]) by smtp.gmail.com with ESMTPSA id w16-20020a1709029a9000b001db60446abdsm5419090plp.262.2024.02.19.22.44.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Feb 2024 22:44:39 -0800 (PST) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, davem@davemloft.net, hannes@cmpxchg.org, herbert@gondor.apana.org.au, linux-crypto@vger.kernel.org, linux-mm@kvack.org, nphamcs@gmail.com, yosryahmed@google.com, zhouchengming@bytedance.com Cc: chriscli@google.com, chrisl@kernel.org, ddstreet@ieee.org, linux-kernel@vger.kernel.org, sjenning@redhat.com, vitaly.wool@konsulko.com, Barry Song Subject: [PATCH v5 1/3] crypto: introduce: acomp_is_async to expose if comp drivers might sleep Date: Tue, 20 Feb 2024 19:44:12 +1300 Message-Id: <20240220064414.262582-2-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240220064414.262582-1-21cnbao@gmail.com> References: <20240220064414.262582-1-21cnbao@gmail.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Barry Song acomp's users might want to know if acomp is really async to optimize themselves. One typical user which can benefit from exposed async stat is zswap. In zswap, zsmalloc is the most commonly used allocator for (and perhaps the only one). For zsmalloc, we cannot sleep while we map the compressed memory, so we copy it to a temporary buffer. By knowing the alg won't sleep can help zswap to avoid the need for a buffer. This shows noticeable improvement in load/store latency of zswap. Signed-off-by: Barry Song Acked-by: Herbert Xu --- include/crypto/acompress.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/include/crypto/acompress.h b/include/crypto/acompress.h index 574cffc90730..80e243611fe2 100644 --- a/include/crypto/acompress.h +++ b/include/crypto/acompress.h @@ -160,6 +160,12 @@ static inline void acomp_request_set_tfm(struct acomp_req *req, req->base.tfm = crypto_acomp_tfm(tfm); } +static inline bool acomp_is_async(struct crypto_acomp *tfm) +{ + return crypto_comp_alg_common(tfm)->base.cra_flags & + CRYPTO_ALG_ASYNC; +} + static inline struct crypto_acomp *crypto_acomp_reqtfm(struct acomp_req *req) { return __crypto_acomp_tfm(req->base.tfm); From patchwork Tue Feb 20 06:44:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13563463 X-Patchwork-Delegate: herbert@gondor.apana.org.au Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6EA3A2EB0A; Tue, 20 Feb 2024 06:44:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708411488; cv=none; b=tB5DVpGUCGV/KMDc91al0imQYHmWa8nJGdEQ7/SbrTWJ3n01ZemQaVRdtxrr5LxiVYNE2ixveIi9jrZs+S2IxMr3bIpzujIKYHwUPZ6x1TMcYxUTZqS3jsY1JqERSpwCPBRZZaLRgIKFhLY++58U3kbN7HJ08Ze8ie6OxL8kHLQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708411488; c=relaxed/simple; bh=QBjD0UNrqJv6Dnwf8bQ7P7DqYnYL3lr2GgKmLuKcx5M=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=R27xHCOOk00ekYHiuA1B0B62+wLN2sbTQnZhuKQkn0gho+q1/LVAICHrN7OgBzsWWe8KAcEfJAL4Dd+RCiMSEXPl0kzSMQa4z3AolnDL9rsxhmtfalM+CmjqoPaEkd+jL9DMuqFdCKHfgYHPpQxW46c2Qwh5v2G+1iPfDMsdeSY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kvzrMOMB; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kvzrMOMB" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1dba94f9201so31412085ad.0; Mon, 19 Feb 2024 22:44:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708411485; x=1709016285; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Z8rNl+BjuOiPHO8BwMbPluxA5iMY9ZQk2ZuevI5ELxs=; b=kvzrMOMBULe+FFGd5lChgE6ZAGN4edqEYZoK6UDbtWJ5/XQRqXX4sSC/lpNk3eqhui ov47XimiIBX6kWHaD2EwO1Vk50E34BMv75EktocCuRatgJZYoGGd5kIQp1vVgh8dn0XG 2W+gmz+W0ne3t3dbekyCxkbn9eGcJmdgrDiDCHqwbkdhmD3zHbT8hppJPDwishz9tiY+ dNRyMRIStGBIDzKL5GaAl17lhYQ+0i9c2hcHU1d43dnz4JEZSdn2GaFlGNWTozJuwX68 3cjxJ03dyRuTMmHOv1Ut2FZwJVRrJuQM2CcHUVAdJph7yEMqBMXYHVG/QdflnYdx7Xwv scrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708411485; x=1709016285; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Z8rNl+BjuOiPHO8BwMbPluxA5iMY9ZQk2ZuevI5ELxs=; b=wCXYxyvFPrubKagHRPVU8oyfufC9DPJ9bGa/m235czbR+/iL3oWWxvN0HvFLC4jihy x7t3R9VAp42EEoX9xqUAL2lcnrl/iqFhGHvUg2JAu5Cvf/gl6AeDJ7dr6uq9/fP3GOn+ GP177MMjY8zKvMuq4FmEk/3mmgziEWJ2vpqTdG2Gqge+6zkpH94rNRGg3tigUbDHycrN JwiS5Nl1pSw/sOfWgjBFU9Lm58H+famoQ+iXY+c7Rz1dTrUpbs6qiMLQrKirGTHnEQen IvaVniafr8EW1/ekOrSZGHR1JTgxV1+SoaKWGi3jlTuKycN1UaWRWGZe8NPpqyq3wnW9 GUVg== X-Forwarded-Encrypted: i=1; AJvYcCWw58r6FLhxLBihEJJgVMGIkSw/kXcc7YHDdF9woU/jUgUS+c1tUjemb8UZg7vhV5UXDlZpfxZ25PsUGsnT+GVEcnfzQ0VoMLvi1A8QjnAxlm8YQuKBs4xmfaSmZl2OUbcAR/lvnw+bSlfP X-Gm-Message-State: AOJu0YwsUCBJrJ1S92+EgGFF9+8HIxx8ORLx+wciWMFD+TL+jWw6ZeUq W/jFPB3jM1KTSD7i6loO911P3ivV2sb0yWR0qTOpk/6L3W42QGoS X-Google-Smtp-Source: AGHT+IGjLI8WrT5H3cLslymf3abuNOE5Ga3w1JP7qFoprv9R9pGvpeHXjEqD7rjwt07BKGzckg9xsg== X-Received: by 2002:a17:903:11cd:b0:1db:82e3:8d7f with SMTP id q13-20020a17090311cd00b001db82e38d7fmr19374961plh.16.1708411485583; Mon, 19 Feb 2024 22:44:45 -0800 (PST) Received: from barry-desktop.hub ([2407:7000:8942:5500:a5bd:9c11:af2b:aecf]) by smtp.gmail.com with ESMTPSA id w16-20020a1709029a9000b001db60446abdsm5419090plp.262.2024.02.19.22.44.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Feb 2024 22:44:45 -0800 (PST) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, davem@davemloft.net, hannes@cmpxchg.org, herbert@gondor.apana.org.au, linux-crypto@vger.kernel.org, linux-mm@kvack.org, nphamcs@gmail.com, yosryahmed@google.com, zhouchengming@bytedance.com Cc: chriscli@google.com, chrisl@kernel.org, ddstreet@ieee.org, linux-kernel@vger.kernel.org, sjenning@redhat.com, vitaly.wool@konsulko.com, Barry Song Subject: [PATCH v5 2/3] mm/zswap: remove the memcpy if acomp is not sleepable Date: Tue, 20 Feb 2024 19:44:13 +1300 Message-Id: <20240220064414.262582-3-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240220064414.262582-1-21cnbao@gmail.com> References: <20240220064414.262582-1-21cnbao@gmail.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Barry Song Most compressors are actually CPU-based and won't sleep during compression and decompression. We should remove the redundant memcpy for them. This patch checks if the algorithm is sleepable by testing the CRYPTO_ALG_ASYNC algorithm flag. Generally speaking, async and sleepable are semantically similar but not equal. But for compress drivers, they are basically equal at least due to the below facts. Firstly, scompress drivers - crypto/deflate.c, lz4.c, zstd.c, lzo.c etc have no sleep. Secondly, zRAM has been using these scompress drivers for years in atomic contexts, and never worried those drivers going to sleep. One exception is that an async driver can sometimes still return synchronously per Herbert's clarification. In this case, we are still having a redundant memcpy. But we can't know if one particular acomp request will sleep or not unless crypto can expose more details for each specific request from offload drivers. Signed-off-by: Barry Song Tested-by: Chengming Zhou Reviewed-by: Nhat Pham Acked-by: Yosry Ahmed Reviewed-by: Chengming Zhou --- mm/zswap.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 51de79aa8659..ef782879291a 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -162,6 +162,7 @@ struct crypto_acomp_ctx { struct crypto_wait wait; u8 *buffer; struct mutex mutex; + bool is_sleepable; }; /* @@ -973,6 +974,7 @@ static int zswap_cpu_comp_prepare(unsigned int cpu, struct hlist_node *node) goto acomp_fail; } acomp_ctx->acomp = acomp; + acomp_ctx->is_sleepable = acomp_is_async(acomp); req = acomp_request_alloc(acomp_ctx->acomp); if (!req) { @@ -1100,7 +1102,7 @@ static void zswap_decompress(struct zswap_entry *entry, struct page *page) mutex_lock(&acomp_ctx->mutex); src = zpool_map_handle(zpool, entry->handle, ZPOOL_MM_RO); - if (!zpool_can_sleep_mapped(zpool)) { + if (acomp_ctx->is_sleepable && !zpool_can_sleep_mapped(zpool)) { memcpy(acomp_ctx->buffer, src, entry->length); src = acomp_ctx->buffer; zpool_unmap_handle(zpool, entry->handle); @@ -1114,7 +1116,7 @@ static void zswap_decompress(struct zswap_entry *entry, struct page *page) BUG_ON(acomp_ctx->req->dlen != PAGE_SIZE); mutex_unlock(&acomp_ctx->mutex); - if (zpool_can_sleep_mapped(zpool)) + if (!acomp_ctx->is_sleepable || zpool_can_sleep_mapped(zpool)) zpool_unmap_handle(zpool, entry->handle); } From patchwork Tue Feb 20 06:44:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13563464 X-Patchwork-Delegate: herbert@gondor.apana.org.au Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD8C95A4DE; Tue, 20 Feb 2024 06:44:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708411494; cv=none; b=Ui6xaMDpbwyjX2+BO/TG9pRV3h5Atn/v30BNVlXNGQqRrYNbJuxmEg8BbLgLYJOQ2ppwXhqUCHscdzXd2vmnF/it6R1PJoRsP2lxi4XGJPWK4Ftxqh7ZrFyPybTRs70fqs7xvXv0y6FyDZxikG2viOYA99w+5FbwIViJiUE0weI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708411494; c=relaxed/simple; bh=OXsYoToIjgeSRYAs9X9E0bo5WCdOjPMMPVgmJGURH78=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Mn5gRDPYSY3JCPOBDBvnKLuN7pSQhmLkZvAJO9buSWlGlSYbANRw1dGam/b4K4y6PLmXfXzj2p1x1a5XVDFfbWgzw5PFk2qzz362ZLjrd+gICmY78fJLYAMMLI/v49paNgiARQ2ofxNnou6qbgbbhyoMG4dBtidCyEtnwr0qCMA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QlVM8ClU; arc=none smtp.client-ip=209.85.216.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QlVM8ClU" Received: by mail-pj1-f45.google.com with SMTP id 98e67ed59e1d1-2997c5fe6abso1897230a91.1; Mon, 19 Feb 2024 22:44:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708411492; x=1709016292; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kW/qQndLj7MYeSMYgAPMliKXg25U647vW1M3U8+Bgew=; b=QlVM8ClU7zuheGbBwmGCYE/VCTle86bL/4mR3GB114bbYj7E3RjvX8kTB8PakVDOe/ GuMvgTIq/Wb3gOPF88bRwV0xcpwUT22ul8InlqpM6nBomP2b4jJdQ+C0y2jO/SCheNYn 8JbjOitnv1qrGSRbjORarp0PiCGhiJHq4uvvp5oUieIW/fM+oYxLpu3qG7HIxuDAr/nU E3XC3Jx4gTRUN8j30vFGvM8smWQpwXmKW1COH0F2iBa+kTD1aVa+xyDt4mHzPb8ed1sE hL/v5Qrmmvob6ZuI5kNmONc/rV5Zg5zoIMkEraDcn4P6tVbXyksp8aEV97iNECOYC6Ab U2mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708411492; x=1709016292; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kW/qQndLj7MYeSMYgAPMliKXg25U647vW1M3U8+Bgew=; b=qZ16NInLwtJOe13q7d7lMy6zp2FL52Lq/xiCww/h4/sun/hKETOPZOcU63ekfWppHp O7z7jygamcTMYVEN9qkcGo6oHjf5ZWrMVbcI7E3PgK0qqg8O+le28R24L8fZIoiapD+/ ptvlRzp0fUDh4cNXTMgjy3WeWw5GUORFy8Qv4noRd6ewh9H/6JVZqPsYt6sC3N4HLZ9o hllXje0sCSnIDxnjBmOjxUIBBkDjLJI5d8IYC8d5iIThK/NXSj9Zo14qMnigs6Lg1FN5 g0L8hTVOVXoMgH+H91WbX+9cW2CRMCrjwKVZLNHUFArioOBaN/X7XjpUPdEeUcla7uKo PCog== X-Forwarded-Encrypted: i=1; AJvYcCUyaJea1y9YLGU9fuTprHJTuqQcBDTWfoDNUHeL3I3WMrW0so/mBmSMI6dh4u692oFLO7NOFkSzIy3d0YAXtEvGA2q+ne/BKtdwItvTNZWc2Z/ipmuIicSYEOk/tHPc+26K42WGVwsxk4Cn X-Gm-Message-State: AOJu0YzqRRZ1OafamOqgCV6K6sqEXaAyybCa+ETBMGR9DA8DRWVYhEiB NPPxuiGLMMHNYXfb0rDjQh8YgTkcnrj4XmZrUIo/VnDKqCke04bO X-Google-Smtp-Source: AGHT+IFy6i7+4k/jye/gmwuX1sSdiY9dP1qorrai79dHWwgMgxT8+F/EE4GSmEybEa4pgheMKtFkLg== X-Received: by 2002:a17:90b:4a08:b0:299:951f:e7de with SMTP id kk8-20020a17090b4a0800b00299951fe7demr4537779pjb.14.1708411492102; Mon, 19 Feb 2024 22:44:52 -0800 (PST) Received: from barry-desktop.hub ([2407:7000:8942:5500:a5bd:9c11:af2b:aecf]) by smtp.gmail.com with ESMTPSA id w16-20020a1709029a9000b001db60446abdsm5419090plp.262.2024.02.19.22.44.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Feb 2024 22:44:51 -0800 (PST) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, davem@davemloft.net, hannes@cmpxchg.org, herbert@gondor.apana.org.au, linux-crypto@vger.kernel.org, linux-mm@kvack.org, nphamcs@gmail.com, yosryahmed@google.com, zhouchengming@bytedance.com Cc: chriscli@google.com, chrisl@kernel.org, ddstreet@ieee.org, linux-kernel@vger.kernel.org, sjenning@redhat.com, vitaly.wool@konsulko.com, Barry Song Subject: [PATCH v5 3/3] crypto: scompress: remove memcpy if sg_nents is 1 Date: Tue, 20 Feb 2024 19:44:14 +1300 Message-Id: <20240220064414.262582-4-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240220064414.262582-1-21cnbao@gmail.com> References: <20240220064414.262582-1-21cnbao@gmail.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Barry Song while sg_nents is 1 which is always true for the current kernel as the only user - zswap is the case, we should remove two big memcpy. Signed-off-by: Barry Song Tested-by: Chengming Zhou --- crypto/scompress.c | 36 +++++++++++++++++++++++++++++------- 1 file changed, 29 insertions(+), 7 deletions(-) diff --git a/crypto/scompress.c b/crypto/scompress.c index b108a30a7600..50a487eac792 100644 --- a/crypto/scompress.c +++ b/crypto/scompress.c @@ -117,6 +117,7 @@ static int scomp_acomp_comp_decomp(struct acomp_req *req, int dir) struct crypto_scomp *scomp = *tfm_ctx; void **ctx = acomp_request_ctx(req); struct scomp_scratch *scratch; + void *src, *dst; unsigned int dlen; int ret; @@ -134,13 +135,25 @@ static int scomp_acomp_comp_decomp(struct acomp_req *req, int dir) scratch = raw_cpu_ptr(&scomp_scratch); spin_lock(&scratch->lock); - scatterwalk_map_and_copy(scratch->src, req->src, 0, req->slen, 0); + if (sg_nents(req->src) == 1) { + src = kmap_local_page(sg_page(req->src)) + req->src->offset; + } else { + scatterwalk_map_and_copy(scratch->src, req->src, 0, + req->slen, 0); + src = scratch->src; + } + + if (req->dst && sg_nents(req->dst) == 1) + dst = kmap_local_page(sg_page(req->dst)) + req->dst->offset; + else + dst = scratch->dst; + if (dir) - ret = crypto_scomp_compress(scomp, scratch->src, req->slen, - scratch->dst, &req->dlen, *ctx); + ret = crypto_scomp_compress(scomp, src, req->slen, + dst, &req->dlen, *ctx); else - ret = crypto_scomp_decompress(scomp, scratch->src, req->slen, - scratch->dst, &req->dlen, *ctx); + ret = crypto_scomp_decompress(scomp, src, req->slen, + dst, &req->dlen, *ctx); if (!ret) { if (!req->dst) { req->dst = sgl_alloc(req->dlen, GFP_ATOMIC, NULL); @@ -152,10 +165,19 @@ static int scomp_acomp_comp_decomp(struct acomp_req *req, int dir) ret = -ENOSPC; goto out; } - scatterwalk_map_and_copy(scratch->dst, req->dst, 0, req->dlen, - 1); + if (dst == scratch->dst) { + scatterwalk_map_and_copy(scratch->dst, req->dst, 0, + req->dlen, 1); + } else { + flush_dcache_page(sg_page(req->dst)); + } } out: + if (src != scratch->src) + kunmap_local(src); + if (dst != scratch->dst) + kunmap_local(dst); + spin_unlock(&scratch->lock); return ret; }