From patchwork Fri Feb 21 09:38:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13985085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DB62C021B3 for ; Fri, 21 Feb 2025 09:39:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB1116B00AC; Fri, 21 Feb 2025 04:39:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D610C280008; Fri, 21 Feb 2025 04:39:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C01D1280004; Fri, 21 Feb 2025 04:39:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A0B076B00AC for ; Fri, 21 Feb 2025 04:39:36 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 63FFD80BB6 for ; Fri, 21 Feb 2025 09:39:36 +0000 (UTC) X-FDA: 83143454352.21.7063D2C Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf03.hostedemail.com (Postfix) with ESMTP id 788712000A for ; Fri, 21 Feb 2025 09:39:34 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=OQwAwACR; spf=pass (imf03.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740130774; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ooRAT30N+1IineDWVp9J6MTWf65J1t58JBJO7W1ETBg=; b=eQLz/rJ+poIURn8QW78eiXBJqt9tw8zpqktarF4RQEKPkj0M9xMYG+BRwW6ZX3W4rN29FT TvG9X/azw+jYXMpU4K49pLqk3TGfW0FXx3LjDjfsO4/EvGUJJQ6jYt+uvWrMw17Rd/0F07 v2AM+vw4OuIpacmdDy416K7CE8FxL2c= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=OQwAwACR; spf=pass (imf03.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740130774; a=rsa-sha256; cv=none; b=1Vxht/hZ752nFUN1ucNC0MMBIhbpvFfJWNxhMRij6sVjFwI9nxI8w/AwS7UVnhZ3iKy2SN 9PpT7M4JmrX7vuTWNwX1SJ28D98EW9E8Bkw7g2hZdBkbfoQrSCFhX9DWGYBNsGfaBS5YQU /NQlSkTg4PkzRTFzHEh4HJ3uvEJn1cQ= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-21f2339dcfdso32869645ad.1 for ; Fri, 21 Feb 2025 01:39:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740130773; x=1740735573; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ooRAT30N+1IineDWVp9J6MTWf65J1t58JBJO7W1ETBg=; b=OQwAwACRw1jZHrE8CZ0M+HrRaxdKNipgWzCacgtbNuHMjR/Azk4srd8KhftoKtUyrh IX1TxSj63m6c2f8xJdgwuZth3I+x1GsjUSDh1+okzawcmpMhD9Ek1erRjBktgC5GLObS TKizPhihHtTCuGYTRHQdefHw/7ZKxba69jNK4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740130773; x=1740735573; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ooRAT30N+1IineDWVp9J6MTWf65J1t58JBJO7W1ETBg=; b=V839gEw/g0aSw17qSpsOMAVmctS/U6sZEXVq22lhPSUolHS5XiLBw0TrGtJxtDapdu DdnE8Ojw0/3wctrmgO0dn/FKkx17JEiQT94+X0mrtJd7XTD4fLblmxWZtHtw+RMgdfZp hhbEW6Zj8mp2CEXbJrQF6/a/j+lzSBN/jYScpr75Qai1Npwa6X+i3v2KLQfWJ34iE7HV Wb0vl4/eKExRDV5R08imnk3QvO4TZojetaI4wuge7ORbvZhgq0+/gLW8PvSifKWe72Ld Y6RfnUuPD1LVPGaz7hY2vHZoLn7ce+fkDDHZoVEd8d0x2aDx/6bWxc0avvDvLImh87nk d1ew== X-Forwarded-Encrypted: i=1; AJvYcCWkgOvEkH7FepsOUC1AZhvGPWLv61RYQSUulha01V8HJakPm56JE8CEyyXiwY3ziXkyl+UZdCNaPQ==@kvack.org X-Gm-Message-State: AOJu0YzzLQiqyyngaG7ZO8hJEZE225rulJJkqCuUPXOaE4C+JZKGAst1 TouvHz0WRwYYcxw4AaW9RELKwcv65anID6lcc+sBciikUQfI/DJmq4I0jwnx0g== X-Gm-Gg: ASbGncvpq40u8rOl8YL1gcO55hcxxYZJtBk2FVCnQfRNoUhZI/O31NhhMsXc9FQIu3m tH2qPi2V0GThuTlg24MAmHK6FMuEI7gAPb235xTS/zwptBoQuKR7F2//6zhXS8BOoNfbB6DqCDt zpViGnHpysdw+dq/D4vCROWhQ2AwydU6JGe58ayL9kBWKQMiMD7rS2HNkqUYcIjqQ8puLDgqUiE F75E0ldhLBVPFeStI1wO+TclPpwtRNvzjrI/394Vc1av9gageiCCDWh/irxGtMb6vQbS3vMRgmE F6SKa+fbG0NV9B5wLKSoV+YMFOo= X-Google-Smtp-Source: AGHT+IGRF3A51Ju1CJymIu4QEKwvzsIx5m8/kDSMhapHrOeLOdsD4MZEno2i5dRKAI8Yq2xbHCHWYA== X-Received: by 2002:a17:903:2311:b0:21f:5063:d3ca with SMTP id d9443c01a7336-2218c5575dbmr109218605ad.16.1740130773425; Fri, 21 Feb 2025 01:39:33 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:f987:e1e:3dbb:2191]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-73242761538sm15686426b3a.139.2025.02.21.01.39.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 21 Feb 2025 01:39:33 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v7 09/17] zram: rework recompression loop Date: Fri, 21 Feb 2025 18:38:02 +0900 Message-ID: <20250221093832.1949691-10-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250221093832.1949691-1-senozhatsky@chromium.org> References: <20250221093832.1949691-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 788712000A X-Stat-Signature: ub347eg9i5fufiu8yw8aaknoji7rkrcn X-Rspamd-Server: rspam03 X-HE-Tag: 1740130774-397378 X-HE-Meta: U2FsdGVkX1+irMgpy0LpmtClfV8sPLQGf/jfFPmnrCj8M5//aBiavsDBXhvDETr0iDgj67cKTyYYPfYLITw/LEO7YCAKSLuZuTDsqhBM4YqJ7/qlWPjVjFIJLn2z+TYpVlcH/InKE+MLOXqth78ZyZjluJ+rLKiZdl7Mq8QJE1UZtc2tAymBiyAT+Hn/7P4/cSS+J5t2pRai+M7Fnu61IUA5yTWDmM837fUiM5nuDJJwn9bsgW1hKQ+QpEjYSGLqaV9mP7wUAt+EUMq39dlxSoAhatFuYnd9xnqRCxDDdYrB9yGG11VIj5J67dv2HeKl/JFCjsRrmUz1J8nkNRRfOHy+gQHgkWeuxuPLFo6Tj5/y4NivPNLxEI535VEJ/obn08Myx625yQCnrcuVroRKaVBLY59QrZv/OJi8dlgi0Mrb+bhkoo1yeAXvLxA1MtsH+l22yiQigMNjQV6REBwXLa3JnUoQsch5z2FVqqVKlD78xOShirKuD9yEche8Ng5pWTgoBta8VDjpxW7zbkZ9WxriP0lmY6C/7PEJMy5f33LeTKbo0sAgdkVogSgy8p4gZSebu5HRtuqrJ9VsLlW8hqkyGjyS3m0StYU7CX1lIQo9tDJ9rgKYS0/JMQhTexJmyx5DQ4SjdhMsqO+Ij8IZpq/mq0Lf25p336tKXsDtttLlfgWXnaWWh6Lm1b786gYDOuTYFWOg+m4AT4/PftW4qs4O+c3I2/kxvN4Pe9/G1FnEYRq0AhWA72XL++Mx48tvnmSY6PbdyfBi/XT5wBh2ObcsvqWP+ao/eaMqXBueuwLJe1xp93FOV0slyXG5V6cjbwJ32+R/lqVUEtyCWSgKjxpFRYLEvvqvIuJ4x3P26Fo0erseUiU2DtzOwNHiVo7lFUAiK3E+LMz6TaYzhNrGvMrmizGGGLtmCstBHnTnm8j/DJKa6jXP2c2NtOGckfS8Ith0hojokUxbhGI6tCU 7VhXf6fF DOO9BsDzSMtHtqf4VEZIFS2mZFHhAEYvJA3NHRzfra4oAvZBFRh2O4YmKLuxs6Qb6pZBwjD1vTUDDeLO9OvZS6ledcGi9tp8U9tJJ2woA6NaDh1ilAtcI8z9kzGXolgilnDl786dEXHZRDlb+Im3cbAJzDrlADrP8fWE9jJFNSoBZSyZ6hHX+BMMk9+sL9VUIUs2IrfJtONamh50l4uZnaw9M+NQqjMkW1+APunRF8oT0lsY+Kz8zmRni2kjuNAcVJgF4nK4GTgIKpKNU1EuU9T0RXU+wHQx4u1d2sEDno6kV51jD4l6rpTaWDw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This reworks recompression loop handling: - set a rule that stream-put NULLs the stream pointer If the loop returns with a non-NULL stream then it's a successfull recompression, otherwise the stream should always be NULL. - do not count the number of recompressions Mark object as incompressible as soon as the algorithm with the highest priority failed to compress that object. - count compression errors as resource usage Even if compression has failed, we still need to bump num_recomp_pages counter. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 53 +++++++++++++---------------------- 1 file changed, 19 insertions(+), 34 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index a1880e621895..814e64f4ee38 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1892,9 +1892,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; - u32 num_recomps = 0; void *src, *dst; - int ret; + int ret = 0; handle_old = zram_get_handle(zram, index); if (!handle_old) @@ -1937,7 +1936,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (!zram->comps[prio]) continue; - num_recomps++; zstrm = zcomp_stream_get(zram->comps[prio]); src = kmap_local_page(page); ret = zcomp_compress(zram->comps[prio], zstrm, @@ -1946,7 +1944,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (ret) { zcomp_stream_put(zstrm); - return ret; + zstrm = NULL; + break; } class_index_new = zs_lookup_class_index(zram->mem_pool, @@ -1956,6 +1955,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { zcomp_stream_put(zstrm); + zstrm = NULL; continue; } @@ -1963,14 +1963,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, break; } - /* - * We did not try to recompress, e.g. when we have only one - * secondary algorithm and the page is already recompressed - * using that algorithm - */ - if (!zstrm) - return 0; - /* * Decrement the limit (if set) on pages we can recompress, even * when current recompression was unsuccessful or did not compress @@ -1980,38 +1972,31 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (*num_recomp_pages) *num_recomp_pages -= 1; - if (class_index_new >= class_index_old) { + /* Compression error */ + if (ret) + return ret; + + if (!zstrm) { /* * Secondary algorithms failed to re-compress the page - * in a way that would save memory, mark the object as - * incompressible so that we will not try to compress - * it again. + * in a way that would save memory. * - * We need to make sure that all secondary algorithms have - * failed, so we test if the number of recompressions matches - * the number of active secondary algorithms. + * Mark the object incompressible if the max-priority + * algorithm couldn't re-compress it. */ - if (num_recomps == zram->num_active_comps - 1) - zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); + if (prio < zram->num_active_comps) + return 0; + zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); return 0; } - /* Successful recompression but above threshold */ - if (threshold && comp_len_new >= threshold) - return 0; - /* - * No direct reclaim (slow path) for handle allocation and no - * re-compression attempt (unlike in zram_write_bvec()) since - * we already have stored that object in zsmalloc. If we cannot - * alloc memory for recompressed object then we bail out and - * simply keep the old (existing) object in zsmalloc. + * We are holding per-CPU stream mutex and entry lock so better + * avoid direct reclaim. Allocation error is not fatal since + * we still have the old object in the mem_pool. */ handle_new = zs_malloc(zram->mem_pool, comp_len_new, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + GFP_NOWAIT | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zstrm); return PTR_ERR((void *)handle_new);