From patchwork Wed Feb 12 06:27:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13971037 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDB72C0219E for ; Wed, 12 Feb 2025 06:32:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 80657280007; Wed, 12 Feb 2025 01:32:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 78F5E280001; Wed, 12 Feb 2025 01:32:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E159280007; Wed, 12 Feb 2025 01:32:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3F141280001 for ; Wed, 12 Feb 2025 01:32:51 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0319C140E11 for ; Wed, 12 Feb 2025 06:32:50 +0000 (UTC) X-FDA: 83110324542.15.A595CDA Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf20.hostedemail.com (Postfix) with ESMTP id 207C11C000C for ; Wed, 12 Feb 2025 06:32:48 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=RqPfZycC; spf=pass (imf20.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.47 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739341969; a=rsa-sha256; cv=none; b=JoeLkiF0mW2KmFBTWuUwqptY2V+KW7X/KcT773+pD6QdUH0GwSDURKD89MJ22S5wBEdSQf pV5SqpnRrmJ1SHA2nyKfFI19mF878r2ix2yI7icrlqReDgenoMuJnkn/qi9gOkGiJmNDQk AQ47dsj3J88nfBggXkn8eFA72+ZwZ7o= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=RqPfZycC; spf=pass (imf20.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.47 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739341969; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cLnpAJqzl8agqjBt4hqeQuPEXwmT47X6Rb4B201CoRE=; b=3FnB86zPPS6uw5Q32wWAekIkDUcIC3dkj2XUNjCBO8lizjTIXbsWkdrczFAvp4/pKeZUGA 8u3RZ1QbQYu2d+IvRV2mdxXnr8OAT58VBvU8emEfrSNlKOB35nIosivoWB6N/BZkC+iFNN fmBFQU5M/ckwcWMwnZ0CSJrPS45RaJM= Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-2fa19e1d027so8560809a91.0 for ; Tue, 11 Feb 2025 22:32:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1739341968; x=1739946768; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cLnpAJqzl8agqjBt4hqeQuPEXwmT47X6Rb4B201CoRE=; b=RqPfZycCoaPc2oGwF6AHV3f1A7D/7ZI1QL3q0n1L4pSjoEJnA4A3kTRJ5rx6IxqO3H CDd2Z+lwxeP1CuD3locCzb0VpygyHmd9f8ScEjbcr9Ja4r8AGue+t/sh9mE9uQqSSLZ/ 8a/euQMWYr96GcULAQg9OnycWsXczHl9YBpBg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739341968; x=1739946768; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cLnpAJqzl8agqjBt4hqeQuPEXwmT47X6Rb4B201CoRE=; b=vXW9DbvlUZl0RjLp5aABqg7MttsBdjl0brS71igkTuKehFC8dRRgebbE5UapE7jvtD mNN6VAP60JL2YfajC+VQMfgDv4xElsKbWR401jxGBjiNrP4CksnXKy+tLEq2EB8i2wr1 EKnVg4faz1pH86lmedbFkxffNxLX7yRDRpjCVS9oOeu4lWEC8S0aMS4sLOvmT3xKn8f2 CDGB9jxEumIizdXVDrMwfcsDtzbhgx1kGsg0tGtsKbaE0swppfVcyyhZ8qv5s7udTcYh pcWWhVSg6B0FsVXlrL8VQRTzGHuw1Jl8SzQKgJMGjQ3UBZQ7muQWvpMbff4TZA1S+GTd 2oXg== X-Forwarded-Encrypted: i=1; AJvYcCX5keCpCslOGcFnAPEAkTR4npdA2gdPG2BaENsAN2PcRqKKWl20vivo9tKrDDK2DiVDX/tSpSYr/w==@kvack.org X-Gm-Message-State: AOJu0YwXGnV2ZujrByuhza9u9wtSCWxLivljshDyl7UXi9dKX5jLE0Is +iPqUFr4Lb0Sm9/RhnBEGqF8Z8C9xq6G3v0fbeVZgsro0XqTx6EF5R/IcvRb7A== X-Gm-Gg: ASbGncts6/qHP1vj52obAL0hBzP7KQfTu55oJHn7gEAPcBTWg3SfBV0v2bdcVUdLR9F UFyIABUQzk2E4DR5BOyp1sdoxbbRJe3PlwdJ60U0Hb0lsFQNeKDXfGgSPnMv9OKWq4KdWK2vCax ujaekgbUPOqsTrMvUorylaRzVKMjqfeFl0hHaqGF444nd4O2ciTjUfbeI9/JgDOuEIw166+bquB Zl2+d4+FygMbxzpCbEBcqbwIeh9jFPVXzGdpFo1a7kgRbm4h0dxgK3K/jyIy4Ihj3tXo74RciQX kgUspl1PGdPlRyHnQw== X-Google-Smtp-Source: AGHT+IHiT+N5GVxrJj2d7+UgbwxnPgo/4IOHt2XM/rWhD+WNLZnaiOqWIvo76vE7u+pn65YNGazKSQ== X-Received: by 2002:a05:6a00:1798:b0:732:24ad:8e08 with SMTP id d2e1a72fcca58-7322c376a72mr3048146b3a.1.1739341967995; Tue, 11 Feb 2025 22:32:47 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:69f5:6852:451e:8142]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-73084e182cdsm5759136b3a.99.2025.02.11.22.32.45 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 11 Feb 2025 22:32:47 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Kairui Song , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v5 09/18] zram: rework recompression loop Date: Wed, 12 Feb 2025 15:27:07 +0900 Message-ID: <20250212063153.179231-10-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.502.g6dc24dfdaf-goog In-Reply-To: <20250212063153.179231-1-senozhatsky@chromium.org> References: <20250212063153.179231-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 207C11C000C X-Stat-Signature: 1p7sj3ijs3bwmtby53jthrt9h44yynij X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1739341968-842124 X-HE-Meta: U2FsdGVkX183JSpD4qXqqmeNeqwt0Bxotsuaa5Wsrq+PA5viRgQ8notssLkCovt2ztloKsJvSpnwtM+VojeAtgxsQa+J0TXHHZrKR4idQpd7qk/I3w9ObZ6uOE0TmsOSatwxoWJutvQpKdgH8ZrQLYV8GGFNwlANgDrpYcMpofn5M3Fr1Uo/TBDpVXWqH92QN/6DY2xeKMpymUYIBMHcvXcZGXcK6UjEusnrbF9p1Qgxa+XDBF8nBS/rzBJp0P/6wsoNniPiTdmV8Lk83lJzmsBNdPF8UhEm5ak+ksDyJUPwQOM42f42pqBgTrtv699rJ2HPVDamUnIJ28f6bqbaBrTtoLOruQVOZk0tR5s05aRYgOSyA1eovAoNpMq7jC8mXAh0uc1q+vtWUbBad2PTgKG7ETmzaLf80l5Tj/PnXUO3yNlfhYBde+Z6R3jJz9aKmZbsAWH7DFK8RUPmIM7ge+l6DauBC1bh8/acDZwBeQHI9AFtcCy2dKgTZO7mevTDkWlw0EjuyoviKV2M+N9qGGj9QDtuk2RAIEeuaZ+75K4xumdLnI8bhuEpjHHuKyFdSB0Bmua/50QSEKYNQXFIkEd9Cld4PPjYLF8pfmhyV5gq3YLq8Jop9r6NpdfPjh0GWw4GyGqvUUYweSsLAVezCOuK0knEfpawiYChZcmpz5QUK9ajeXemHMAfWI2TGt/Tc74QdzSAQBxY5MaDiHxnDxlG/xeVyrwaSwUaqgjcqW+Mw7G7hiASx+fAxNaHQ9JTRnMPJ+O7OFzA6cmQm/dqXHuhReLABP3Pe4rojXDSiErq/RvjgH8741CSwlhHrz6jSGS71Y53vzDldLI/RmbNBPCm1EH1GFGo6xC3ytIz204cY1bMpNuStA+Ghe3kGcyhw2FKSgX5iZNfLMHpGSwSndD62UXp9FWRh+DRAa8YNddbh2uJytdbBoyKDFeCsuBq9cCafZuhjehBVnfch7w V8RjiRqv YlgCE24x8MiAO0M6P10MlIcJEYnXz/Iyv3bLaMkAWShPEGZ6LYCf0OoPb+QFEYTuS34hiCLfSEyGajvj7NLEN02PPU02NvDMrcCGUa6tkYRyZ4lANgA23Y6zPlqNgYD7vwzwMcqFImwBAHstrYj2gP9LoAsvJnXRLgG2Zt2FvCsnb5wDf5hJLG9qyJP+nXuO0HYq5GbfpGK38jSY6ifHWHOGb7lQ8Vn8GzY2xdBEkY+w+yb11BCYCWrJUMmad3+fRl3EIKVeU39033447LwP1S8/WgXBL7Et9JdgiLYbSPHT7PVY2vWCRQ6dRMQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This reworks recompression loop handling: - set a rule that stream-put NULLs the stream pointer If the loop returns with a non-NULL stream then it's a successfull recompression, otherwise the stream should always be NULL. - do not count the number of recompressions Mark object as incompressible as soon as the algorithm with the highest priority failed to compress that object. - count compression errors as resource usage Even if compression has failed, we still need to bump num_recomp_pages counter. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 53 +++++++++++++---------------------- 1 file changed, 19 insertions(+), 34 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 31bdf5e0ff74..7c4c296181a8 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1887,9 +1887,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; - u32 num_recomps = 0; void *src, *dst; - int ret; + int ret = 0; handle_old = zram_get_handle(zram, index); if (!handle_old) @@ -1932,7 +1931,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (!zram->comps[prio]) continue; - num_recomps++; zstrm = zcomp_stream_get(zram->comps[prio]); src = kmap_local_page(page); ret = zcomp_compress(zram->comps[prio], zstrm, @@ -1941,7 +1939,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (ret) { zcomp_stream_put(zstrm); - return ret; + zstrm = NULL; + break; } class_index_new = zs_lookup_class_index(zram->mem_pool, @@ -1951,6 +1950,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { zcomp_stream_put(zstrm); + zstrm = NULL; continue; } @@ -1958,14 +1958,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, break; } - /* - * We did not try to recompress, e.g. when we have only one - * secondary algorithm and the page is already recompressed - * using that algorithm - */ - if (!zstrm) - return 0; - /* * Decrement the limit (if set) on pages we can recompress, even * when current recompression was unsuccessful or did not compress @@ -1975,38 +1967,31 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (*num_recomp_pages) *num_recomp_pages -= 1; - if (class_index_new >= class_index_old) { + /* Compression error */ + if (ret) + return ret; + + if (!zstrm) { /* * Secondary algorithms failed to re-compress the page - * in a way that would save memory, mark the object as - * incompressible so that we will not try to compress - * it again. + * in a way that would save memory. * - * We need to make sure that all secondary algorithms have - * failed, so we test if the number of recompressions matches - * the number of active secondary algorithms. + * Mark the object incompressible if the max-priority + * algorithm couldn't re-compress it. */ - if (num_recomps == zram->num_active_comps - 1) - zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); + if (prio < zram->num_active_comps) + return 0; + zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); return 0; } - /* Successful recompression but above threshold */ - if (threshold && comp_len_new >= threshold) - return 0; - /* - * No direct reclaim (slow path) for handle allocation and no - * re-compression attempt (unlike in zram_write_bvec()) since - * we already have stored that object in zsmalloc. If we cannot - * alloc memory for recompressed object then we bail out and - * simply keep the old (existing) object in zsmalloc. + * We are holding per-CPU stream mutex and entry lock so better + * avoid direct reclaim. Allocation error is not fatal since + * we still have the old object in the mem_pool. */ handle_new = zs_malloc(zram->mem_pool, comp_len_new, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + GFP_NOWAIT | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zstrm); return PTR_ERR((void *)handle_new);