From patchwork Fri Feb 14 04:50:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13974471 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7472BC02198 for ; Fri, 14 Feb 2025 04:53:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F5886B009E; Thu, 13 Feb 2025 23:53:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 07F5E6B009F; Thu, 13 Feb 2025 23:53:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E3A64280001; Thu, 13 Feb 2025 23:53:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C2BB56B009E for ; Thu, 13 Feb 2025 23:53:11 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 75ECE161D20 for ; Fri, 14 Feb 2025 04:53:11 +0000 (UTC) X-FDA: 83117330982.19.07BD38E Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf24.hostedemail.com (Postfix) with ESMTP id 9F84E180007 for ; Fri, 14 Feb 2025 04:53:09 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=D7wM0s3E; spf=pass (imf24.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739508789; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=l/xbb64LFlIEuFMcJ/r7e+VdeaEJuKPUYJ75LZ96nQ8=; b=hgriNYCWwsNQZ76IcU86xM8H5Kglxdweqr2ej/J2iXF3sjJ9JYVvUIowJd0mdj5baPEFhM ohdqNXaWT0lIQy5KNe0Ccntvv+vHwZaxpWHDqtaIzyMDNreYm+uwjn2prit5NMUN+gj4N6 cQXdP07JIih3xZLlRHD2c9SNd0ydNhM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739508789; a=rsa-sha256; cv=none; b=4+T5JvjhnLgNztiu0RtD7voMq+6PV10b+yKleyQZ40r7cePXWEDPtuOEf9yi1Hhe1tvw9p xkxPztDyfbvq2jNGPX0tXGHc7KSA33z2mjyvMnBzWGCbSHgoyJH9c6tK4NeHwHTBNUB7RT OnO+ngRxVXqeHJ8oXN5ZOshEFsnRg4g= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=D7wM0s3E; spf=pass (imf24.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-21c2f1b610dso38141585ad.0 for ; Thu, 13 Feb 2025 20:53:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739508788; x=1740113588; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=l/xbb64LFlIEuFMcJ/r7e+VdeaEJuKPUYJ75LZ96nQ8=; b=D7wM0s3ExnDqFwnZv3bdorcxgo9OXwAb3YawsjOpC7BNJWh7g+jxwbbjomWwjEVl1a f/t7etPeqkhBzqcFrDEQxlJMDU7GtDYLI3Sz/gBZfS7/0HDN/iYUKGWMpKfwel8mWesM 7qmRCx8BKpnFNb/H6xw+qAsP6GaF1emMjIbCU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739508788; x=1740113588; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=l/xbb64LFlIEuFMcJ/r7e+VdeaEJuKPUYJ75LZ96nQ8=; b=Qjp95kpxJ2PI1Iba0u+KZhy3aQMUCkx9ub98mY333lurCKRQTvQY6lvxVfBvizAy71 +ltiHm6mAufTTuHiM3YsM+gJN9/9C6Bn2R4zjrP4g3t8vHixHCoIPyNir9f+AdqNBAhK TUa8DLGKJ23OVG5Y0i+hp5N3Ufibg8oG1IFWJyRMcBlGCqrQrcT5BVoWqFB4BqZH4KfM 27LY//tnyGZOtXyfrsY/H4sJzWfimxMCMt/1OzTk6Aa2Qnkh4BuvxCErPOn7Ey07VQVf 2vCLn752QQUj7PcKo8NZUryj9YUvv4ypxarzJxiDilJU837MdJwsO2yGNTwIpjlwAAEM ZVvQ== X-Forwarded-Encrypted: i=1; AJvYcCUT1TU4Du4MUvGau0kCxNm2xDHIcuVgIhdaXVDm3CBtlB0JICpLBW5cFJkxkcBWFL+4Ek4bMMk0XQ==@kvack.org X-Gm-Message-State: AOJu0YxlM9PQPbF8yqUgtoXVEqrmW4Y3/2syUNH+DeKYhMjTozIb5V0Y /r6AK56IyMpbWETyglB+tQqFdhdXqhcKBFlv+rZL7E3H6FUlYsONMeMgq1EWIA== X-Gm-Gg: ASbGncsWh1hyLok0KqdNFMNYvyT4O4gRa6mCnec35CxIs1OQmuKIoGiVndJfGGtoe71 L6rZ6qTCtzt4X+JuJZx7QSbiMQliyky61JH1/U34u5806iQ8k10J5V8UPpxpC/hSo25DNobnITd 4/JTcIXuYeGDopz3oqPI5bxSvlQ7HcfZi4mp+rpZO74+9CJgx5aXbEMyFvabZZwmwg39tMyD5IA 3DGFovBSpHL3KMSzUjDB8/qZc/XJVlcsy+DTYn8zfVxpDxnR5sxDkiH2GxoHHXSmeX3cFKiRU61 4/pNA2yPr4ZB6oMOKA== X-Google-Smtp-Source: AGHT+IH47sqL4VstyP3CQTqYF0OisImdj6UBrruloe47Ytl0rlHucdtsjUQ25uJQt/qj5N2jpQbI3w== X-Received: by 2002:a17:902:dad2:b0:21f:baa:80c1 with SMTP id d9443c01a7336-220d2364ac7mr109012865ad.53.1739508788597; Thu, 13 Feb 2025 20:53:08 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:942d:9291:22aa:8126]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d556e15esm21001265ad.190.2025.02.13.20.53.06 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 20:53:08 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v6 09/17] zram: rework recompression loop Date: Fri, 14 Feb 2025 13:50:21 +0900 Message-ID: <20250214045208.1388854-10-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250214045208.1388854-1-senozhatsky@chromium.org> References: <20250214045208.1388854-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 9F84E180007 X-Rspamd-Server: rspam07 X-Stat-Signature: uxsngbfmsszid8q3f88gf9uczmtg8sc9 X-HE-Tag: 1739508789-205127 X-HE-Meta: U2FsdGVkX1/xuS1O2vEwdYw9GwYBmlaDsaOp+qKgsIK4jhst0ZUFYm99qAFFTeXcaSBs4t8SXC5em9BYLtFOhKpm7KdHhrRu3g61Y6mZtrbSrflnDWlDCMhc4uK8EGq7dnpxKXPewmuRMC099Fg4ZDL6vIWP4Cg3lIR4fRBuSy3l99kYw1JjW4jar0GcVMx6E1vj/YmET3RxQhHgCrlJ+m+WUxpScV72AE2nsnexQwuQKuFa/piNh25VgsxtNK0LgmHORDRHIw9c9+uDyR7d3ZxO93QhC6T9Gy2K3g1vOpZIu+b710ovzxElQzs4TPlMO9zzm/Y2MMNs/WQlJ8g9mEyR1+8gn4zGPjFEf2lyylZcLuq0RkDA6TKSDj1uNz8+kVDFZSpTkxyjyP8zIqb8if9Vjw7acupm3IQcsT0mefUQcvrhF8Kq/FI9zJaud+FlmDDGV4SPfrmARLz0zd7COCUncrby0gJEmzrznEB3rbkKOfL9FysIF+wZofyNoDRLJPWvA2LsweTIujD67XDtRQaKYvojQOYqf1fY9/MKd1/FGAC11Ge1Pf/ajXp7l0Br/2z/GzjP39PzGbZa4sbfGnADciD7f0++sciThZtwvp72qOiXKSV05OqEq6vw5w1M7rlTvbOrLM6S+0jSItZwERsOCHYMew6NVWTHCjc1nuTh24SrA87/NgHtsFAzqFda9bq8ufusH6kmVRKh3y1VzGAN+TzVCYbGkUSoIWlPfmDf6phhMjewqDBItvrClM9VVBu7NuxUIflqCnSHbEVCkNGRrkBQG1ZEkL7IVRHrGujeJENRZAnHwnIUjvgKLANO0fq9ojoHD3tZOd7KBLAppqYPxfYzGfVtdqdxqefRN2WBUnmXTJnbYBFf2Gc8oNYSmrDyL2vRYkebMbKnWgXXAHyjR/ryJi1obGEeaxxOj7LV9bfXInpuTkPwkinNQXIxhBhAs4hht9jcEWqJcqw BmZpJBKF 8TMx3jMXDYQBqBpVrlKAI7YixI79QAhFMZ4+F2gV0vYjLoWLVtibGbBrNe7wAxOk2tfi5zjrYuzvqzeZDU0jCFhyZKF+dTF54CLBKeKzPE66ElJb3tfiRU8/VjZ89VXYQm9HrI5EmS2HVCNVJwUy8NuoEg7V4R9zeSFS+Q8uVu+6flrsOR3Ak2UlDxaKl+KfECa2Gz4jvwD/2AkxID0yyxP5A0JeVtfUvRrT7qq8Q3heBNaOXNJxs5TJ53S4UTDTMDc6tiwAG4ldjs5fEnbu9NzHy+XivFNazUidGEOBPYU4IiUN4n+xFEHjpHQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This reworks recompression loop handling: - set a rule that stream-put NULLs the stream pointer If the loop returns with a non-NULL stream then it's a successfull recompression, otherwise the stream should always be NULL. - do not count the number of recompressions Mark object as incompressible as soon as the algorithm with the highest priority failed to compress that object. - count compression errors as resource usage Even if compression has failed, we still need to bump num_recomp_pages counter. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 53 +++++++++++++---------------------- 1 file changed, 19 insertions(+), 34 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 5f8e963bd513..2e46ea542ecd 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1929,9 +1929,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; - u32 num_recomps = 0; void *src, *dst; - int ret; + int ret = 0; handle_old = zram_get_handle(zram, index); if (!handle_old) @@ -1974,7 +1973,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (!zram->comps[prio]) continue; - num_recomps++; zstrm = zcomp_stream_get(zram->comps[prio]); src = kmap_local_page(page); ret = zcomp_compress(zram->comps[prio], zstrm, @@ -1983,7 +1981,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (ret) { zcomp_stream_put(zstrm); - return ret; + zstrm = NULL; + break; } class_index_new = zs_lookup_class_index(zram->mem_pool, @@ -1993,6 +1992,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { zcomp_stream_put(zstrm); + zstrm = NULL; continue; } @@ -2000,14 +2000,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, break; } - /* - * We did not try to recompress, e.g. when we have only one - * secondary algorithm and the page is already recompressed - * using that algorithm - */ - if (!zstrm) - return 0; - /* * Decrement the limit (if set) on pages we can recompress, even * when current recompression was unsuccessful or did not compress @@ -2017,38 +2009,31 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (*num_recomp_pages) *num_recomp_pages -= 1; - if (class_index_new >= class_index_old) { + /* Compression error */ + if (ret) + return ret; + + if (!zstrm) { /* * Secondary algorithms failed to re-compress the page - * in a way that would save memory, mark the object as - * incompressible so that we will not try to compress - * it again. + * in a way that would save memory. * - * We need to make sure that all secondary algorithms have - * failed, so we test if the number of recompressions matches - * the number of active secondary algorithms. + * Mark the object incompressible if the max-priority + * algorithm couldn't re-compress it. */ - if (num_recomps == zram->num_active_comps - 1) - zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); + if (prio < zram->num_active_comps) + return 0; + zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); return 0; } - /* Successful recompression but above threshold */ - if (threshold && comp_len_new >= threshold) - return 0; - /* - * No direct reclaim (slow path) for handle allocation and no - * re-compression attempt (unlike in zram_write_bvec()) since - * we already have stored that object in zsmalloc. If we cannot - * alloc memory for recompressed object then we bail out and - * simply keep the old (existing) object in zsmalloc. + * We are holding per-CPU stream mutex and entry lock so better + * avoid direct reclaim. Allocation error is not fatal since + * we still have the old object in the mem_pool. */ handle_new = zs_malloc(zram->mem_pool, comp_len_new, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + GFP_NOWAIT | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zstrm); return PTR_ERR((void *)handle_new);