From patchwork Thu Feb 27 04:35:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13993740 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CDDDC197BF for ; Thu, 27 Feb 2025 04:37:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C8C028000C; Wed, 26 Feb 2025 23:37:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 97961280005; Wed, 26 Feb 2025 23:37:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8197C28000C; Wed, 26 Feb 2025 23:37:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 62635280005 for ; Wed, 26 Feb 2025 23:37:29 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id DD6D4801E6 for ; Thu, 27 Feb 2025 04:37:28 +0000 (UTC) X-FDA: 83164465776.17.8AF30C8 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf20.hostedemail.com (Postfix) with ESMTP id 0DE2B1C0010 for ; Thu, 27 Feb 2025 04:37:26 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Zy6Vjk4p; spf=pass (imf20.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.170 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740631047; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=s6NLD2jL1GvsoHkQKXZoOwiQJj4KiHrs9duVpBtevTk=; b=h2SwHRgUxqwcpxsuZ5QB6fhV+9yb4l5w7FieGh9Qr0WYvHM0Z6NB8dwGUf6/353doHIW3U uOZCIKW1TKydJ7GbAC94WXOZclLgpTnhwYshz2LaZtF5JH5aHq4ApfaMsSuAkGUsjrXMR5 t2+KUeZj2gBPDVb4QTpHc3zoX62NS/8= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Zy6Vjk4p; spf=pass (imf20.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.170 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740631047; a=rsa-sha256; cv=none; b=O/XC3xzMwwnGMktZJw+uYyKW2TEUIbFn3gocs2WnmmAY0JmubseNE27cyvUpQELglbQTH0 2iET5kNu4WNmjI4ajACYtwo1y8ENI7XDTFOvaZo1ZbECygtVu70chE7ITfWZXandABKO6v rY9pTqFin+z2LRG3xrMsabhLZOTib+s= Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2211acda7f6so9980775ad.3 for ; Wed, 26 Feb 2025 20:37:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740631046; x=1741235846; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=s6NLD2jL1GvsoHkQKXZoOwiQJj4KiHrs9duVpBtevTk=; b=Zy6Vjk4piQJWnDa5p/2Hykr23qYV+WxALu9EoWHrwTBhJoWn25gvvUGxtQD9sI+JHW iDrierL6cIVd5BtWL9kYuDmXtaBjKzdYmXS9WE0NpaachiY2zpTxzjks4dNLvspXBko/ NVygkU/vgROAzfI2wbep420FShgZAsgCw9KUY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740631046; x=1741235846; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=s6NLD2jL1GvsoHkQKXZoOwiQJj4KiHrs9duVpBtevTk=; b=Flqmo0dT/46cMWLs/YKj0AgNBaP/NJKMZlpzlz4yGpbiTId8XYXFiWCnMuNequJ/GT SCiPKbKTH3ejxwBfLObBld4RG1uDmN+mtAh/RE1E6JKAYXpkyXVBJAvYZHc5YihpaLV7 7l3JuCf/yAVbmZZiLlFNmPmsMSB2ro9zjz3o+t1oPjSrbGvEJE3VA80H8fQ46D2cL5kf r3ndFExGA5jcQkjRSTqJuSgrBPEJX7+4yrZlesp+L1EQIP8iNL3sZs7netB4xOFadOSE HyTGTkTOG95oPrfyE3MtFOvzj/8DsKNAO1pxvrqpjNrsDPWhzAxWlLHE2Z9pzAe7TfnS K3ew== X-Forwarded-Encrypted: i=1; AJvYcCWRu1XhskA50rSVuGKDzHUYl5ZNiiDmI9bQigKhw+29BC5C0c08DJ7d2OZm3JIuxf6tRtnv+KbuSw==@kvack.org X-Gm-Message-State: AOJu0YxRnnRjq8QDP190OSSo4AVcTnxC3T9WDr1/5ccBZ/9wK1TlQVWn MNGEPxDNBMG2VqNAEs4y3zJBte6nWwupx2bOS+dNvak7R5vSnBKhOTpFogCIGA== X-Gm-Gg: ASbGncs49v4K6gVIjDMEV5TkpHBB4NXJzTbRdo2N8uLRtM1Ip7TRhMdPy1AM4NnMvlK RsiWayyJls0J7nFMWRr0xEHGdiZPgsj6TGydKnr9VtYH9uQQwoY/hfVirLerk5nmrMhobis0Cv3 pAeuxAoaXFLrnBQoei57tAg08unN/29l9CqV14M3E4EHwC7Yle4AqZYexd0+x0JQeS+NXzwiqj/ W/PK2KnwXLl4p9YB/AdewpOYyEkxzmTVRhj3Bn5+KOvg+aC+f6ncWp/H8Z4wpRLLxqi37q9wPOn Z65Qfl5Xn6yZH3b52WKUqd1erQjC X-Google-Smtp-Source: AGHT+IE7XcAdc9vNrlMBc7Oz2yCA4ADvv2oX4w4TjS94Lh2tcodEasjQb7dK+y4SdRXQWBQQi/bf9w== X-Received: by 2002:a17:903:22cd:b0:216:2bd7:1c2f with SMTP id d9443c01a7336-2232009991bmr68248845ad.18.1740631045980; Wed, 26 Feb 2025 20:37:25 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:a9c0:1bc1:74e3:3e31]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-223501fa918sm5062645ad.69.2025.02.26.20.37.23 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Feb 2025 20:37:25 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v9 10/19] zram: rework recompression loop Date: Thu, 27 Feb 2025 13:35:28 +0900 Message-ID: <20250227043618.88380-11-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog In-Reply-To: <20250227043618.88380-1-senozhatsky@chromium.org> References: <20250227043618.88380-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0DE2B1C0010 X-Stat-Signature: crn8t8rgimgm16o7hga6fugzt44eyaec X-HE-Tag: 1740631046-491145 X-HE-Meta: U2FsdGVkX1/HLDW3oCryWQGPidkQYi1++832p5Wc9ufcBepmxqxCLoNkQoioAfa2kKTZwoGGroS/vK8s43ziRmInUFhcidRV/M3DxwJQ5luraESyI6lq+C+uKhWUlZzoHHSalZBbQPvNcXE/TSa9zOioyWbikuF9kh/BSc7cblzJWtaZg990V5HMQD/7oVyJBgcUwzTdZpCqYGBBFKT/c/MymVyNSViDGbsJdlOCxEGJt8U8srgP/rfHhmgjULmYnogsNZvv7G5qzJNqS2ckug1kpGnDGpfXHeCW+0/RnSVJKB6p9KBxlLfV91ioOBCq0/o/WcGPGUX6rbPq5TGSntfSZfmBYWwewk9lktHwd1bWsxV2r2Af7IGALnvmmzLv6IYJaIBgRzMBV+9VZeT9h33rKdsDbhqvLPCmLEl3+Ha+7euJSiXkwRx7YWxCmWM1D/iTil1ddRRB18c0Xf6ng9BqsrUoWMSApKk2eVWafUezZWwKLsI25q3/qOsuPMeZGWZQfIr+wzuK42w5GEYmyfocTdse91HOOuv5vctGRT+nfSDwRKwH6PnMkmadH3iwP0K5fGuTAi0nIzRxUN21N2CktN1hoHz/lbRGQnzIv3tOM6DY4OGRCMnaa0P3/a9MHWyUivUwtf/4Erzy7SqTlXC8hv8rRm18nc4S//Zn2OytSIhu4rU31D0qzyJf/pb46joemwI3+3bJ8soo2BLWt+7b7wfSiRAiCQdtplj6Krwi8Y7QvX32o/E7V/CeiRYjs+WkDblnNE55XUGPwUsLLTjx1GyUslXsRW/5sq3HW3/Y6Mi5tqjCiVLwaxx4s243zOsGXbnJ7OZwG4pNbUj1oTJY1NbATXwg+iiH2jw/18Jb5UO8Gjz0pjKBMUjUz9KDhgORM313UGBdGMupar6bAqeC9fSjWLKGb1u2/aHdv5tEmm/aI6CQqFGwVx11SVFGA2I9EoPomgenYLCXZiw fCiqNKAh 5h6XCNbWtn8b35gJpg+oCKOTzP/J8KVeFVOsanVJ5L3B3Rls00WIM2I1indc1omWnDD6vE4lVHiha9qrP8bS6U3Z9MRJv5518mg/6jM9vpOcL9UN6AwoLBuVQtrPvMgDmd8FWpOkpCKEA7nteT3ss1AEzs8pZEvLgbtoisyoJMigWL/MUPY7/woTnQTxWqsm3esA7uj2XkF/fj8dw9dWNTQA8hykLnsdWdqAj4sxE8C3UwltfewygYKCjuvXIPSxNS1gIW4+QGBRXPL5Qg1DsceLCGI8slWJb6jMhFwH7DhYHrYFX0gOjDZ6o2g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This reworks recompression loop handling: - set a rule that stream-put NULLs the stream pointer If the loop returns with a non-NULL stream then it's a successful recompression, otherwise the stream should always be NULL. - do not count the number of recompressions Mark object as incompressible as soon as the algorithm with the highest priority failed to compress that object. - count compression errors as resource usage Even if compression has failed, we still need to bump num_recomp_pages counter. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 54 +++++++++++++---------------------- 1 file changed, 20 insertions(+), 34 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 3a23dfed9542..6755ca90f737 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1888,9 +1888,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; - u32 num_recomps = 0; void *src, *dst; - int ret; + int ret = 0; handle_old = zram_get_handle(zram, index); if (!handle_old) @@ -1933,7 +1932,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (!zram->comps[prio]) continue; - num_recomps++; zstrm = zcomp_stream_get(zram->comps[prio]); src = kmap_local_page(page); ret = zcomp_compress(zram->comps[prio], zstrm, @@ -1942,7 +1940,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (ret) { zcomp_stream_put(zstrm); - return ret; + zstrm = NULL; + break; } class_index_new = zs_lookup_class_index(zram->mem_pool, @@ -1952,6 +1951,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { zcomp_stream_put(zstrm); + zstrm = NULL; continue; } @@ -1959,14 +1959,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, break; } - /* - * We did not try to recompress, e.g. when we have only one - * secondary algorithm and the page is already recompressed - * using that algorithm - */ - if (!zstrm) - return 0; - /* * Decrement the limit (if set) on pages we can recompress, even * when current recompression was unsuccessful or did not compress @@ -1976,38 +1968,32 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (*num_recomp_pages) *num_recomp_pages -= 1; - if (class_index_new >= class_index_old) { + /* Compression error */ + if (ret) + return ret; + + if (!zstrm) { /* * Secondary algorithms failed to re-compress the page - * in a way that would save memory, mark the object as - * incompressible so that we will not try to compress - * it again. + * in a way that would save memory. * - * We need to make sure that all secondary algorithms have - * failed, so we test if the number of recompressions matches - * the number of active secondary algorithms. + * Mark the object incompressible if the max-priority + * algorithm couldn't re-compress it. */ - if (num_recomps == zram->num_active_comps - 1) - zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); + if (prio < zram->num_active_comps) + return 0; + zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); return 0; } - /* Successful recompression but above threshold */ - if (threshold && comp_len_new >= threshold) - return 0; - /* - * No direct reclaim (slow path) for handle allocation and no - * re-compression attempt (unlike in zram_write_bvec()) since - * we already have stored that object in zsmalloc. If we cannot - * alloc memory for recompressed object then we bail out and - * simply keep the old (existing) object in zsmalloc. + * We are holding per-CPU stream mutex and entry lock so better + * avoid direct reclaim. Allocation error is not fatal since + * we still have the old object in the mem_pool. */ handle_new = zs_malloc(zram->mem_pool, comp_len_new, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + GFP_NOIO | __GFP_NOWARN | + __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zstrm); return PTR_ERR((void *)handle_new);