From patchwork Thu Jan 30 11:10:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13954482 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 398FAC0218D for ; Thu, 30 Jan 2025 11:11:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA2666B00C3; Thu, 30 Jan 2025 06:11:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A51D06B00C4; Thu, 30 Jan 2025 06:11:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F2DA280286; Thu, 30 Jan 2025 06:11:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6C64B6B00C3 for ; Thu, 30 Jan 2025 06:11:22 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A3E2546211 for ; Thu, 30 Jan 2025 11:11:21 +0000 (UTC) X-FDA: 83063851962.04.AA510EF Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) by imf05.hostedemail.com (Postfix) with ESMTP id 96A7410000D for ; Thu, 30 Jan 2025 11:11:19 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=RLaMGSdg; spf=pass (imf05.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.49 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738235479; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vG/tk94hewr73CT+Z4xbU1wydP7qgFMVbNXS/fz+Vpw=; b=FU2NzA94B5cIW4ZoS9Ajfgeb0EUAHzcYccse1CzVyazaMLOfGZyqRC2vAFuhw1Y0tw/kmj 2GXmWoky6dBvhtqoNPiSZGfwgGrPF4Zw+SspUVSbQ7fdaDxP5/b0tAQ2FbTVOyykU8NyJE oJIfLZgl+Uh8lUX6lQHnEsAvE3PXDqc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738235479; a=rsa-sha256; cv=none; b=C9Pn8a1yBGLhaEyeS+t8ZudrcN631yHaj0HVvY/PUPQuCdzF9zUnnMGg0US4k4ZjUvPGOH yiM4CPDbCy5sH5oJjDQwyqVZdJlGnColqJMsS0ikITHhuEAAdRaEyQbgtbyf9qZdinc5UO qLfKDx7SR/5PDSBxtL6RgWjKRG3JRGg= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=RLaMGSdg; spf=pass (imf05.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.49 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pj1-f49.google.com with SMTP id 98e67ed59e1d1-2ee8e8e29f6so787284a91.0 for ; Thu, 30 Jan 2025 03:11:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738235478; x=1738840278; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vG/tk94hewr73CT+Z4xbU1wydP7qgFMVbNXS/fz+Vpw=; b=RLaMGSdgsRUfj+VJsuIGcrPhgscFpbu6kHpOECASqyofLsVo+zWEYk157ihslQg8Kw 2UNMJmCJF4+j5awqBcHL3V0nnzMf/gOiTC3YuY4W+wlvh/NHNVNhKKOMLIn1c2AGlU0D Zj9ZWy2YsxDNlcEgCNse0l/aKobaahJ98gwxs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738235478; x=1738840278; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vG/tk94hewr73CT+Z4xbU1wydP7qgFMVbNXS/fz+Vpw=; b=dEiKbc4sVyj8ObBjdij6xQSZ7kctnmFsKOm0iu5ykVeUyauRIJTK3CnF2cB/3PcP6v fQ/2IRH/Lg/AOCNu5D8Cle91G5Ss9fdfpZGQNK4bxeo6FC5GkHtB9MoKGAJWPnYIha4T wclPRoToJni5nHtZ93e4hhyDrEMGxCWoQueRIlF853yHpRs8t0X8N35FBej1GtXZPPHB aSOgviUxxsbLcsVecl/Ixib3QwEp4MTnujJitp1fJbsdcgt5ZekEE5/8Rqk/mSlfrcSq uK6lVyR83vqzwSRH8yHQy/AdMDaUfLywZmhN3XjkkT1Qn6vwsKyf2jm/LVhdpvVatRrT 6wzA== X-Forwarded-Encrypted: i=1; AJvYcCX7eThOHn7EQr8Kk7Bqhv+Tz+75XSaqr6E7zhUzjZyK77LfFoj95K6T6pNw+IUBEP0k6RbL5X8fwQ==@kvack.org X-Gm-Message-State: AOJu0YwcPnLUtQFoXcJ20BNCQ9Cew4Snil+hAylLma2+Zv4lOANEqFng WhvAB878o8+k7oWJYuIHGS1nouhIp9DSEN6rTUfj7XQjy/R0TPnAU93EW7vjDw== X-Gm-Gg: ASbGnctwhibKL+kDBZNKmjkZJJrlJ3ZB/VsRM5l9IBF1XhD8qG3JQoHgqaePXRUc4JO 9Tk8EEb0QjEOPi0rJVUudBCWXHWnlw+Xh0vqESJKq9r8j0lstDz9uUy65tTSS1Fdhqf4Ci/KaaX GASYWYIhRVvgRHEldwNbw/5UL+JYqX8Wl38uEEFN3iEJcR/wdudthj2DzVk9G6tmKQ/rsjgOvQM M7PUh1eEUPhIWJQJa54jl4Yk9gIGtD/MEaqjeu+j0t2QXaABmWEI4snTtzed2yerQaDWVJGc3K+ DlVx7y4ChmjUkxRN X-Google-Smtp-Source: AGHT+IEII0bxfv0lWA3uPhG032lc2h9qT5EqknbZ0568rgvm+cDL3+X00otQuH/5VTn7k1d91oV5cA== X-Received: by 2002:a17:90b:3a0c:b0:2ee:c9b6:4c42 with SMTP id 98e67ed59e1d1-2f83abff391mr10977943a91.16.1738235478355; Thu, 30 Jan 2025 03:11:18 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:d4ce:e744:f46b:4fb]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2f83bd0ce4dsm3638761a91.27.2025.01.30.03.11.16 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Jan 2025 03:11:18 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv3 01/11] zram: switch to non-atomic entry locking Date: Thu, 30 Jan 2025 20:10:46 +0900 Message-ID: <20250130111105.2861324-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250130111105.2861324-1-senozhatsky@chromium.org> References: <20250130111105.2861324-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Stat-Signature: mex65pnzrjn1yjadi19xn1e3r48yh84n X-Rspam-User: X-Rspamd-Queue-Id: 96A7410000D X-Rspamd-Server: rspam03 X-HE-Tag: 1738235479-545594 X-HE-Meta: U2FsdGVkX196/+MM+EY8rotfCctEnPad1/8/hoPx+oAPf6Bn9bB7/ZsF0UtmhYDv9xJ39JIY4T0A+astvCqmb7jeYI1oRaAGHBcBzxKohx/3GKqbCGX69GdJQxX/8hLf+KaAJhxz3NNEIN5wbrT+ldyYULeEgvg/6VgZy3X629fRVTJbVpsPtRf/1kbXdKfdCJu+0Z2tdEOWEnEqK9ZRv6dvOCDb9hh34KmkhpWFtNXW+vyfu/jinrtX5ZrOVdfc6U3kl2w0tu1PEAE61u/euS6vXqvJPooFG2WhN9QAyGGZAcA2Qr+vLUJtvBHJkIe/y306pPfe5DdJlo6rg/nMJP1JYZjwRllFm9gL8W+sGraxggsaX9DOo1LibHg+d3Pu+Y2dcNeeSycs/Hru/XTpZUXkS3PRvN1dCtg33mQRpsMERMuLgLR6DThsTAAAvOgLfJJEha2xQaHc6Jq/gAGCHXhhOBowJeHDL/1QMC/2wuwxPh4JH6btBWPZuzfz8Ggxs9iLnHmRbH+f4lMnDCz/NDzV8Y6QHdIVhjLsCP0ltZIWJtaLUqlnjtRDsjV+FjtA83MvqDnQ3a5tRYuCnowJ88jfqyZA2hkVziINBTLA+mqeTrSQG2oXVMIjZx6Kt5DgzF4QqdanpLKuUwz6IrrCPFGMqwo3RHncRqaxFRtOgtT/ZoxmLYdJ189wGJkUvhALrSo6AB7mCC6/z3zcamqknsOq6vEBkmkEC00Gb6TtcKJV4dA/6xgzw2oK2ngYJw8WhiSztoGhdL+jRrv1gYKA1zUVhGJpxdILV8xAvNhMslSNXzs9+CSWwNUGntG8LSf2h+PPmQvLywFbF50MOZ4c8BwriUNoNq+bxsJKAMsdhZh6JLajR2lInJmIgxmSNlkcKMJ0u4GE+wma17Eup5Gbf/nGDuLg2sx1fMg8RDGC8cynhFTRXoFhrzMkR6PsCAjZvMybqM04ACTtSIqq1LQ nT7yFzom fZ9f/SzYMWJBYxmPG8XAGi1sK0ESYXu09gtkQEfj7B1Oi4Bdbm47eDi70p0ufteeUfSvJS8JARFhvTgAYuagulei+mUP9LUVfBAUu09TZVZF7W9k6BrSpHzEn2+o45hMx4a11MACtvL7bO0ydLBybMLnOTB//Z6DbljTVfc3L+OUr34Ci166MVYUFi6CG/nT8jrof2zwWnuBx+uYDQy2RwgtaIiPXmBalWpJrmCOknDoZ0BLhQLGYmbgLwacJ+gCnm3hxoEiU9M1qksZhxzj8BVNVSRDySsZnftPs8XubShWATqqVYU01J4e3Zw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Concurrent modifications of meta table entries is now handled by per-entry spin-lock. This has a number of shortcomings. First, this imposes atomic requirements on compression backends. zram can call both zcomp_compress() and zcomp_decompress() under entry spin-lock, which implies that we can use only compression algorithms that don't schedule/sleep/wait during compression and decompression. This, for instance, makes it impossible to use some of the ASYNC compression algorithms (H/W compression, etc.) implementations. Second, this can potentially trigger watchdogs. For example, entry re-compression with secondary algorithms is performed under entry spin-lock. Given that we chain secondary compression algorithms and that some of them can be configured for best compression ratio (and worst compression speed) zram can stay under spin-lock for quite some time. Do not use per-entry spin-locks and instead convert it to an atomic_t variable which open codes reader-writer type of lock. This permits preemption from slot_lock section. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 148 ++++++++++++++++++++-------------- drivers/block/zram/zram_drv.h | 7 +- 2 files changed, 91 insertions(+), 64 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 9f5020b077c5..a8d22ae2a066 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -58,19 +58,50 @@ static void zram_free_page(struct zram *zram, size_t index); static int zram_read_from_zspool(struct zram *zram, struct page *page, u32 index); -static int zram_slot_trylock(struct zram *zram, u32 index) +static bool zram_slot_try_write_lock(struct zram *zram, u32 index) { - return spin_trylock(&zram->table[index].lock); + atomic_t *lock = &zram->table[index].lock; + int old = ZRAM_ENTRY_UNLOCKED; + + return atomic_try_cmpxchg(lock, &old, ZRAM_ENTRY_WRLOCKED); } -static void zram_slot_lock(struct zram *zram, u32 index) +static void zram_slot_write_lock(struct zram *zram, u32 index) { - spin_lock(&zram->table[index].lock); + atomic_t *lock = &zram->table[index].lock; + int old = atomic_read(lock); + + do { + if (old != ZRAM_ENTRY_UNLOCKED) { + cond_resched(); + old = atomic_read(lock); + continue; + } + } while (!atomic_try_cmpxchg(lock, &old, ZRAM_ENTRY_WRLOCKED)); +} + +static void zram_slot_write_unlock(struct zram *zram, u32 index) +{ + atomic_set(&zram->table[index].lock, ZRAM_ENTRY_UNLOCKED); +} + +static void zram_slot_read_lock(struct zram *zram, u32 index) +{ + atomic_t *lock = &zram->table[index].lock; + int old = atomic_read(lock); + + do { + if (old == ZRAM_ENTRY_WRLOCKED) { + cond_resched(); + old = atomic_read(lock); + continue; + } + } while (!atomic_try_cmpxchg(lock, &old, old + 1)); } -static void zram_slot_unlock(struct zram *zram, u32 index) +static void zram_slot_read_unlock(struct zram *zram, u32 index) { - spin_unlock(&zram->table[index].lock); + atomic_dec(&zram->table[index].lock); } static inline bool init_done(struct zram *zram) @@ -93,7 +124,6 @@ static void zram_set_handle(struct zram *zram, u32 index, unsigned long handle) zram->table[index].handle = handle; } -/* flag operations require table entry bit_spin_lock() being held */ static bool zram_test_flag(struct zram *zram, u32 index, enum zram_pageflags flag) { @@ -229,9 +259,9 @@ static void release_pp_slot(struct zram *zram, struct zram_pp_slot *pps) { list_del_init(&pps->entry); - zram_slot_lock(zram, pps->index); + zram_slot_write_lock(zram, pps->index); zram_clear_flag(zram, pps->index, ZRAM_PP_SLOT); - zram_slot_unlock(zram, pps->index); + zram_slot_write_unlock(zram, pps->index); kfree(pps); } @@ -394,11 +424,11 @@ static void mark_idle(struct zram *zram, ktime_t cutoff) * * And ZRAM_WB slots simply cannot be ZRAM_IDLE. */ - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); if (!zram_allocated(zram, index) || zram_test_flag(zram, index, ZRAM_WB) || zram_test_flag(zram, index, ZRAM_SAME)) { - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); continue; } @@ -410,7 +440,7 @@ static void mark_idle(struct zram *zram, ktime_t cutoff) zram_set_flag(zram, index, ZRAM_IDLE); else zram_clear_flag(zram, index, ZRAM_IDLE); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); } } @@ -709,7 +739,7 @@ static int scan_slots_for_writeback(struct zram *zram, u32 mode, INIT_LIST_HEAD(&pps->entry); - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); if (!zram_allocated(zram, index)) goto next; @@ -731,7 +761,7 @@ static int scan_slots_for_writeback(struct zram *zram, u32 mode, place_pp_slot(zram, ctl, pps); pps = NULL; next: - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); } kfree(pps); @@ -822,7 +852,7 @@ static ssize_t writeback_store(struct device *dev, } index = pps->index; - zram_slot_lock(zram, index); + zram_slot_read_lock(zram, index); /* * scan_slots() sets ZRAM_PP_SLOT and relases slot lock, so * slots can change in the meantime. If slots are accessed or @@ -833,7 +863,7 @@ static ssize_t writeback_store(struct device *dev, goto next; if (zram_read_from_zspool(zram, page, index)) goto next; - zram_slot_unlock(zram, index); + zram_slot_read_unlock(zram, index); bio_init(&bio, zram->bdev, &bio_vec, 1, REQ_OP_WRITE | REQ_SYNC); @@ -860,7 +890,7 @@ static ssize_t writeback_store(struct device *dev, } atomic64_inc(&zram->stats.bd_writes); - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); /* * Same as above, we release slot lock during writeback so * slot can change under us: slot_free() or slot_free() and @@ -882,7 +912,7 @@ static ssize_t writeback_store(struct device *dev, zram->bd_wb_limit -= 1UL << (PAGE_SHIFT - 12); spin_unlock(&zram->wb_limit_lock); next: - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); release_pp_slot(zram, pps); cond_resched(); @@ -1001,7 +1031,7 @@ static ssize_t read_block_state(struct file *file, char __user *buf, for (index = *ppos; index < nr_pages; index++) { int copied; - zram_slot_lock(zram, index); + zram_slot_read_lock(zram, index); if (!zram_allocated(zram, index)) goto next; @@ -1019,13 +1049,13 @@ static ssize_t read_block_state(struct file *file, char __user *buf, ZRAM_INCOMPRESSIBLE) ? 'n' : '.'); if (count <= copied) { - zram_slot_unlock(zram, index); + zram_slot_read_unlock(zram, index); break; } written += copied; count -= copied; next: - zram_slot_unlock(zram, index); + zram_slot_read_unlock(zram, index); *ppos += 1; } @@ -1455,33 +1485,31 @@ static void zram_meta_free(struct zram *zram, u64 disksize) static bool zram_meta_alloc(struct zram *zram, u64 disksize) { - size_t num_pages, index; + size_t num_ents, index; - num_pages = disksize >> PAGE_SHIFT; - zram->table = vzalloc(array_size(num_pages, sizeof(*zram->table))); + num_ents = disksize >> PAGE_SHIFT; + zram->table = vzalloc(array_size(num_ents, sizeof(*zram->table))); if (!zram->table) - return false; + goto error; zram->mem_pool = zs_create_pool(zram->disk->disk_name); - if (!zram->mem_pool) { - vfree(zram->table); - zram->table = NULL; - return false; - } + if (!zram->mem_pool) + goto error; if (!huge_class_size) huge_class_size = zs_huge_class_size(zram->mem_pool); - for (index = 0; index < num_pages; index++) - spin_lock_init(&zram->table[index].lock); + for (index = 0; index < num_ents; index++) + atomic_set(&zram->table[index].lock, ZRAM_ENTRY_UNLOCKED); + return true; + +error: + vfree(zram->table); + zram->table = NULL; + return false; } -/* - * To protect concurrent access to the same index entry, - * caller should hold this table index entry's bit_spinlock to - * indicate this index entry is accessing. - */ static void zram_free_page(struct zram *zram, size_t index) { unsigned long handle; @@ -1602,17 +1630,17 @@ static int zram_read_page(struct zram *zram, struct page *page, u32 index, { int ret; - zram_slot_lock(zram, index); + zram_slot_read_lock(zram, index); if (!zram_test_flag(zram, index, ZRAM_WB)) { /* Slot should be locked through out the function call */ ret = zram_read_from_zspool(zram, page, index); - zram_slot_unlock(zram, index); + zram_slot_read_unlock(zram, index); } else { /* * The slot should be unlocked before reading from the backing * device. */ - zram_slot_unlock(zram, index); + zram_slot_read_unlock(zram, index); ret = read_from_bdev(zram, page, zram_get_handle(zram, index), parent); @@ -1655,10 +1683,10 @@ static int zram_bvec_read(struct zram *zram, struct bio_vec *bvec, static int write_same_filled_page(struct zram *zram, unsigned long fill, u32 index) { - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); zram_set_flag(zram, index, ZRAM_SAME); zram_set_handle(zram, index, fill); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); atomic64_inc(&zram->stats.same_pages); atomic64_inc(&zram->stats.pages_stored); @@ -1693,11 +1721,11 @@ static int write_incompressible_page(struct zram *zram, struct page *page, kunmap_local(src); zs_unmap_object(zram->mem_pool, handle); - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); zram_set_flag(zram, index, ZRAM_HUGE); zram_set_handle(zram, index, handle); zram_set_obj_size(zram, index, PAGE_SIZE); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); atomic64_add(PAGE_SIZE, &zram->stats.compr_data_size); atomic64_inc(&zram->stats.huge_pages); @@ -1718,9 +1746,9 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) bool same_filled; /* First, free memory allocated to this slot (if any) */ - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); zram_free_page(zram, index); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); mem = kmap_local_page(page); same_filled = page_same_filled(mem, &element); @@ -1790,10 +1818,10 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); zs_unmap_object(zram->mem_pool, handle); - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); zram_set_handle(zram, index, handle); zram_set_obj_size(zram, index, comp_len); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); /* Update stats */ atomic64_inc(&zram->stats.pages_stored); @@ -1850,7 +1878,7 @@ static int scan_slots_for_recompress(struct zram *zram, u32 mode, INIT_LIST_HEAD(&pps->entry); - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); if (!zram_allocated(zram, index)) goto next; @@ -1871,7 +1899,7 @@ static int scan_slots_for_recompress(struct zram *zram, u32 mode, place_pp_slot(zram, ctl, pps); pps = NULL; next: - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); } kfree(pps); @@ -2162,7 +2190,7 @@ static ssize_t recompress_store(struct device *dev, if (!num_recomp_pages) break; - zram_slot_lock(zram, pps->index); + zram_slot_write_lock(zram, pps->index); if (!zram_test_flag(zram, pps->index, ZRAM_PP_SLOT)) goto next; @@ -2170,7 +2198,7 @@ static ssize_t recompress_store(struct device *dev, &num_recomp_pages, threshold, prio, prio_max); next: - zram_slot_unlock(zram, pps->index); + zram_slot_write_unlock(zram, pps->index); release_pp_slot(zram, pps); if (err) { @@ -2217,9 +2245,9 @@ static void zram_bio_discard(struct zram *zram, struct bio *bio) } while (n >= PAGE_SIZE) { - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); zram_free_page(zram, index); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); atomic64_inc(&zram->stats.notify_free); index++; n -= PAGE_SIZE; @@ -2248,9 +2276,9 @@ static void zram_bio_read(struct zram *zram, struct bio *bio) } flush_dcache_page(bv.bv_page); - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); zram_accessed(zram, index); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); bio_advance_iter_single(bio, &iter, bv.bv_len); } while (iter.bi_size); @@ -2278,9 +2306,9 @@ static void zram_bio_write(struct zram *zram, struct bio *bio) break; } - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); zram_accessed(zram, index); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); bio_advance_iter_single(bio, &iter, bv.bv_len); } while (iter.bi_size); @@ -2321,13 +2349,13 @@ static void zram_slot_free_notify(struct block_device *bdev, zram = bdev->bd_disk->private_data; atomic64_inc(&zram->stats.notify_free); - if (!zram_slot_trylock(zram, index)) { + if (!zram_slot_try_write_lock(zram, index)) { atomic64_inc(&zram->stats.miss_free); return; } zram_free_page(zram, index); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); } static void zram_comp_params_reset(struct zram *zram) diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index db78d7c01b9a..3436ddf8ab23 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -15,7 +15,6 @@ #ifndef _ZRAM_DRV_H_ #define _ZRAM_DRV_H_ -#include #include #include @@ -28,7 +27,6 @@ #define ZRAM_SECTOR_PER_LOGICAL_BLOCK \ (1 << (ZRAM_LOGICAL_BLOCK_SHIFT - SECTOR_SHIFT)) - /* * ZRAM is mainly used for memory efficiency so we want to keep memory * footprint small and thus squeeze size and zram pageflags into a flags @@ -58,13 +56,14 @@ enum zram_pageflags { __NR_ZRAM_PAGEFLAGS, }; -/*-- Data structures */ +#define ZRAM_ENTRY_UNLOCKED 0 +#define ZRAM_ENTRY_WRLOCKED (-1) /* Allocated for each disk page */ struct zram_table_entry { unsigned long handle; unsigned int flags; - spinlock_t lock; + atomic_t lock; #ifdef CONFIG_ZRAM_TRACK_ENTRY_ACTIME ktime_t ac_time; #endif From patchwork Thu Jan 30 11:10:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13954483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14BDFC0218A for ; Thu, 30 Jan 2025 11:11:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B8286B00C4; Thu, 30 Jan 2025 06:11:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 91986280286; Thu, 30 Jan 2025 06:11:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 776076B00C7; Thu, 30 Jan 2025 06:11:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 50F266B00C4 for ; Thu, 30 Jan 2025 06:11:26 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id BF8C8A0BFB for ; Thu, 30 Jan 2025 11:11:25 +0000 (UTC) X-FDA: 83063852130.24.CD05ED6 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf17.hostedemail.com (Postfix) with ESMTP id D0A3440006 for ; Thu, 30 Jan 2025 11:11:23 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=GZj5BYiD; spf=pass (imf17.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738235483; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UEH++QnSGY0Fi6Z8VZI2JREfdhZluHMkf5kZ0a7B1uQ=; b=YIRsPtmcZ5THegpgaAvIdiIY6uKNZeeDaZZjkC08KEosXkTIZuNzQ0dgC5YGzFs3pNHjNd az2m6usY6Tf2qllNDhXcXHESqj0vbkRB3UrSNSpdYEETYKqrGb/USivcRHNfCal17IlMCl ecGtLiulddgsca+/LeXL8xlH3oZg9Vs= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=GZj5BYiD; spf=pass (imf17.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738235483; a=rsa-sha256; cv=none; b=1WeJ9Tjpo5rAF7KEMCWmxByajEg/zmoC1HzEiiaYWTZuIsY7f5SPjLCX3gdAGz8K+5D94e I4tAkgvGHx0fSii8bh9d2LcRXCh51bg7zBEU/m3gR+w5kPMO59FnefTYmREqT+RkjmrvnL D/fjzokxbsD1dkoITLujqxb1WHZqN9A= Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2163b0c09afso9876085ad.0 for ; Thu, 30 Jan 2025 03:11:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738235482; x=1738840282; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UEH++QnSGY0Fi6Z8VZI2JREfdhZluHMkf5kZ0a7B1uQ=; b=GZj5BYiDT2OJgXTKL0+oDfoE02suL6N3+DV8Aencjx2E/Z4O1zUUQFIqq6L0ta7bnb Av8HMSgroqUjT7y78i/zfmYsJ/F8SF9eAdpipXHDe7qDH1Zs8Urk30yFRXfW11/RcMWk 3YtDIBjeQ6YOf6QP4qN2KAoECHv9/N7p7F67Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738235482; x=1738840282; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UEH++QnSGY0Fi6Z8VZI2JREfdhZluHMkf5kZ0a7B1uQ=; b=OaCs21jKoX1VB5OCGPxEjkZ8Rm/0aYukjWzI5hIWkyrTz31iNlz47A/CNQo90MxnCQ dGCBBsaph86RsQpVDZn9H5m57C5zezxOc52PTVCwOQCcl9LnWoDdtQeOABIWupz9msMn 2KKGaMVcvSPn8D+2px3Qad7efF+D1J9Y3j8bZy4+ngvwZjXvX46jc0A6ZEDi8oatxn8w B9niGouMsHXzY/oiFbVdoSv5pZufloqwixGw+ojnEwprQekgC1rTxV3IxLBqEj5mzrea IqEsbv+ciM/udZC51VaAu6NMIVqqzJB0MN8K62jVZ0H2h3V2KdvfCVs27x0upSofNk4+ 4KpA== X-Forwarded-Encrypted: i=1; AJvYcCUf023fDn48ZobH94bsLmh2bzOJJV9nbGNNLPbwL+uHrVUVihMr88JWBPvHX6LxznasL9uu13rB7Q==@kvack.org X-Gm-Message-State: AOJu0Ywmwr+Fpkw0IbKpPJOibRbECiYejJEcftGahbViS/Nx8/N5fpzh m7CabYXf0Ilu6jNjKTetxH0ZciYv8xiDxQQhMUGwjZ30yPNFRjk6/Ap2DdXmfJJIGYOrVlATh2M = X-Gm-Gg: ASbGncuw60DBQtU7DA02xnSHgCm83KweLH5F5LamUCt7s9vroU4j4YWctaCI7ZbEWUp L3PK+rVJf4PPtyZYP1w2pUX+ay2WgMPmGv8Ml5Rt+JRLIl8soHl8xnXByVy0MkIsTXiQ0dwfCT3 quuIQuU7frNnOhAJJy3gOQR9xtzi91Vrre0CBVvUp0jnsXft2HMtQGeYFUaBpzHiMPxb0sKa6p1 jhqgp9V6+Xrg7Badgp98NgGufO1q5WsjpC6l3tGE4ng/Zp7lVM9T1MhEntYFa9CN6osU0sh+SeO yShLZ1EibnFn+/Ff X-Google-Smtp-Source: AGHT+IH+TH8HeSlHJqfOK6VsPbvyML0ro0yaIHBMXXZ5hNOFlr6JBn/DYjgqdQ5lAb/0oKdeeTUdYw== X-Received: by 2002:a05:6a21:7882:b0:1e7:6f82:3217 with SMTP id adf61e73a8af0-1ed7a5c2316mr9568435637.3.1738235482594; Thu, 30 Jan 2025 03:11:22 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:d4ce:e744:f46b:4fb]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72fe69ba47esm1183433b3a.96.2025.01.30.03.11.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Jan 2025 03:11:22 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv3 02/11] zram: do not use per-CPU compression streams Date: Thu, 30 Jan 2025 20:10:47 +0900 Message-ID: <20250130111105.2861324-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250130111105.2861324-1-senozhatsky@chromium.org> References: <20250130111105.2861324-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: D0A3440006 X-Stat-Signature: kcwxh3krgkq6zdn6fssypiki3i554yec X-HE-Tag: 1738235483-494916 X-HE-Meta: U2FsdGVkX1/izRdfevLbJ46/iZ9qcTxA3c7juAqJPpfJM4W3kWH/h2nOCI1jFv7XiSDTWCnhD5ro+j1Nol5NU5DRETr2HRPnKDf/BDCAynhrkbZFSMal5EnTtD3OtUOjhiJ0ZCGeD/fJFk6og5kik0Ond+0sWA5dT83pEe+Q7JzSbCwclTf64xVEeHBkgQF62l7BOGn0x3KVkHGLJz8AWsJoshQ5euyJpwEudyKOFvSADa7SSMbWFuq+4lqygU30G5bEWhx4RonNnwZwpyRRx67FAe9k1qcMCQkcYuudVTw6EDLzJJyv/D65cWr8bNKLT7YPiKRpygHTRuRF9FGS4hODvQTkIrdV/JqLHeNyXDpEg9xiK5qUNbpnUvqFa3tCMSj0rjNiqwx5Z+Z/dB+kh+FCHI6Kxfp1SxbKLSVSUbdQXzwK4hkzSdIx7XfUHnZ4fJQftZNaMDI48wRiqwg0RrgW7o+M2YeSNLPLfPGDzwvCFmn0gfGoSjY1X7Vvw1lx+yZaJJJhNiYIupGIU1DUFp7Z1H01TsF5vapqg0slvAT999waegN8qJAN1CuVCEurRZkVGolw+/CFVH8irym0xNhIbmzHWXh4xgt0BwuDv4MsrmQt4c0FppXM/8H8XJM1H1PjsTCB/WL13oNROKG4GCdrfCp+vojDDlfu5lIRkCRKLe+wPaYyjUoYZJBiWP07UDiSXOb3mly6L/0nyNQlv1q26NTrxWb6vcYk3ewRmtjCciD8okxV3dV6UIJ8XrtBa7kzq5kVZm1FPZr6/gZHvF4BJsHam++BAJS5X8BcUDsg9OKFdF3tn/zkz77xxCxOcnbDLOFuCLVRH0p63f6F6yAQj6CuTUWiPzYYbh80aZ8JnfuQADWgZQZtodzo4E516IUusBDPJfrUSK7U1R4j3Z7Oh1skVMt+MuzwQIx43Q8WAhYsmKVBFidOnTLqQRd3q1TrBXm13VJFMeuzCcC P8yFHwc3 ljBgQ/FVN9x6pk0zLw3eayIQGwX41ShDKtwGjV/7+GvlGhXftBmpodfsPy9ecvIdOaiaOtxXrQOV/ds0b8wRT4Ix9fMxM9oOKvSSX9wtjkXNtONORq6MKMvNKuZZGbVpbclCDStzQu/I2wKbrY8XXrzehiJ/yVP/kDpJCnEZr6ulUVUz++I96NRunEWwdXLcbawPxjwtZSRfgSX2lFU/rSW/tDifmxi85kcgM7DqRtOwYi8SgpWt8OL5umJ/HdbfZ+yDYD2I/fSn+2LDKXx7JDYO08W2H9RazLonF4ladbPY2Il2QdJ3bNBIbaw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Similarly to per-entry spin-lock per-CPU compression streams also have a number of shortcoming. First, per-CPU stream access has to be done from a non-preemptible (atomic) section, which imposes the same atomicity requirements on compression backends as entry spin-lock do and makes it impossible to use algorithms that can schedule/wait/sleep during compression and decompression. Second, per-CPU streams noticeably increase memory usage (actually more like wastage) of secondary compression streams. The problem is that secondary compression streams are allocated per-CPU, just like the primary streams are. Yet we never use more that one secondary stream at a time, because recompression is a single threaded action. Which means that remaining num_online_cpu() - 1 streams are allocated for nothing, and this is per-priority list (we can have several secondary compression algorithms). Depending on the algorithm this may lead to a significant memory wastage, in addition each stream also carries a workmem buffer (2 physical pages). Instead of per-CPU streams, maintain a list of idle compression streams and allocate new streams on-demand (something that we used to do many years ago). So that zram read() and write() become non-atomic and ease requirements on the compression algorithm implementation. This also means that we now should have only one secondary stream per-priority list. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 164 +++++++++++++++++++--------------- drivers/block/zram/zcomp.h | 17 ++-- drivers/block/zram/zram_drv.c | 29 +++--- include/linux/cpuhotplug.h | 1 - 4 files changed, 109 insertions(+), 102 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index bb514403e305..982c769d5831 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -6,7 +6,7 @@ #include #include #include -#include +#include #include #include @@ -43,31 +43,40 @@ static const struct zcomp_ops *backends[] = { NULL }; -static void zcomp_strm_free(struct zcomp *comp, struct zcomp_strm *zstrm) +static void zcomp_strm_free(struct zcomp *comp, struct zcomp_strm *strm) { - comp->ops->destroy_ctx(&zstrm->ctx); - vfree(zstrm->buffer); - zstrm->buffer = NULL; + comp->ops->destroy_ctx(&strm->ctx); + vfree(strm->buffer); + kfree(strm); } -static int zcomp_strm_init(struct zcomp *comp, struct zcomp_strm *zstrm) +static struct zcomp_strm *zcomp_strm_alloc(struct zcomp *comp) { + struct zcomp_strm *strm; int ret; - ret = comp->ops->create_ctx(comp->params, &zstrm->ctx); - if (ret) - return ret; + strm = kzalloc(sizeof(*strm), GFP_KERNEL); + if (!strm) + return NULL; + + INIT_LIST_HEAD(&strm->entry); + + ret = comp->ops->create_ctx(comp->params, &strm->ctx); + if (ret) { + kfree(strm); + return NULL; + } /* - * allocate 2 pages. 1 for compressed data, plus 1 extra for the - * case when compressed size is larger than the original one + * allocate 2 pages. 1 for compressed data, plus 1 extra in case if + * compressed data is larger than the original one. */ - zstrm->buffer = vzalloc(2 * PAGE_SIZE); - if (!zstrm->buffer) { - zcomp_strm_free(comp, zstrm); - return -ENOMEM; + strm->buffer = vzalloc(2 * PAGE_SIZE); + if (!strm->buffer) { + zcomp_strm_free(comp, strm); + return NULL; } - return 0; + return strm; } static const struct zcomp_ops *lookup_backend_ops(const char *comp) @@ -109,13 +118,59 @@ ssize_t zcomp_available_show(const char *comp, char *buf) struct zcomp_strm *zcomp_stream_get(struct zcomp *comp) { - local_lock(&comp->stream->lock); - return this_cpu_ptr(comp->stream); + struct zcomp_strm *strm; + + might_sleep(); + + while (1) { + spin_lock(&comp->strm_lock); + if (!list_empty(&comp->idle_strm)) { + strm = list_first_entry(&comp->idle_strm, + struct zcomp_strm, + entry); + list_del(&strm->entry); + spin_unlock(&comp->strm_lock); + return strm; + } + + /* cannot allocate new stream, wait for an idle one */ + if (comp->avail_strm >= num_online_cpus()) { + spin_unlock(&comp->strm_lock); + wait_event(comp->strm_wait, + !list_empty(&comp->idle_strm)); + continue; + } + + /* allocate new stream */ + comp->avail_strm++; + spin_unlock(&comp->strm_lock); + + strm = zcomp_strm_alloc(comp); + if (strm) + break; + + spin_lock(&comp->strm_lock); + comp->avail_strm--; + spin_unlock(&comp->strm_lock); + wait_event(comp->strm_wait, !list_empty(&comp->idle_strm)); + } + + return strm; } -void zcomp_stream_put(struct zcomp *comp) +void zcomp_stream_put(struct zcomp *comp, struct zcomp_strm *strm) { - local_unlock(&comp->stream->lock); + spin_lock(&comp->strm_lock); + if (comp->avail_strm <= num_online_cpus()) { + list_add(&strm->entry, &comp->idle_strm); + spin_unlock(&comp->strm_lock); + wake_up(&comp->strm_wait); + return; + } + + comp->avail_strm--; + spin_unlock(&comp->strm_lock); + zcomp_strm_free(comp, strm); } int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, @@ -148,61 +203,19 @@ int zcomp_decompress(struct zcomp *comp, struct zcomp_strm *zstrm, return comp->ops->decompress(comp->params, &zstrm->ctx, &req); } -int zcomp_cpu_up_prepare(unsigned int cpu, struct hlist_node *node) -{ - struct zcomp *comp = hlist_entry(node, struct zcomp, node); - struct zcomp_strm *zstrm; - int ret; - - zstrm = per_cpu_ptr(comp->stream, cpu); - local_lock_init(&zstrm->lock); - - ret = zcomp_strm_init(comp, zstrm); - if (ret) - pr_err("Can't allocate a compression stream\n"); - return ret; -} - -int zcomp_cpu_dead(unsigned int cpu, struct hlist_node *node) -{ - struct zcomp *comp = hlist_entry(node, struct zcomp, node); - struct zcomp_strm *zstrm; - - zstrm = per_cpu_ptr(comp->stream, cpu); - zcomp_strm_free(comp, zstrm); - return 0; -} - -static int zcomp_init(struct zcomp *comp, struct zcomp_params *params) -{ - int ret; - - comp->stream = alloc_percpu(struct zcomp_strm); - if (!comp->stream) - return -ENOMEM; - - comp->params = params; - ret = comp->ops->setup_params(comp->params); - if (ret) - goto cleanup; - - ret = cpuhp_state_add_instance(CPUHP_ZCOMP_PREPARE, &comp->node); - if (ret < 0) - goto cleanup; - - return 0; - -cleanup: - comp->ops->release_params(comp->params); - free_percpu(comp->stream); - return ret; -} - void zcomp_destroy(struct zcomp *comp) { - cpuhp_state_remove_instance(CPUHP_ZCOMP_PREPARE, &comp->node); + struct zcomp_strm *strm; + + while (!list_empty(&comp->idle_strm)) { + strm = list_first_entry(&comp->idle_strm, + struct zcomp_strm, + entry); + list_del(&strm->entry); + zcomp_strm_free(comp, strm); + } + comp->ops->release_params(comp->params); - free_percpu(comp->stream); kfree(comp); } @@ -229,7 +242,12 @@ struct zcomp *zcomp_create(const char *alg, struct zcomp_params *params) return ERR_PTR(-EINVAL); } - error = zcomp_init(comp, params); + INIT_LIST_HEAD(&comp->idle_strm); + init_waitqueue_head(&comp->strm_wait); + spin_lock_init(&comp->strm_lock); + + comp->params = params; + error = comp->ops->setup_params(comp->params); if (error) { kfree(comp); return ERR_PTR(error); diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h index ad5762813842..62330829db3f 100644 --- a/drivers/block/zram/zcomp.h +++ b/drivers/block/zram/zcomp.h @@ -3,10 +3,10 @@ #ifndef _ZCOMP_H_ #define _ZCOMP_H_ -#include - #define ZCOMP_PARAM_NO_LEVEL INT_MIN +#include + /* * Immutable driver (backend) parameters. The driver may attach private * data to it (e.g. driver representation of the dictionary, etc.). @@ -31,7 +31,7 @@ struct zcomp_ctx { }; struct zcomp_strm { - local_lock_t lock; + struct list_head entry; /* compression buffer */ void *buffer; struct zcomp_ctx ctx; @@ -60,16 +60,15 @@ struct zcomp_ops { const char *name; }; -/* dynamic per-device compression frontend */ struct zcomp { - struct zcomp_strm __percpu *stream; + struct list_head idle_strm; + spinlock_t strm_lock; + u32 avail_strm; + wait_queue_head_t strm_wait; const struct zcomp_ops *ops; struct zcomp_params *params; - struct hlist_node node; }; -int zcomp_cpu_up_prepare(unsigned int cpu, struct hlist_node *node); -int zcomp_cpu_dead(unsigned int cpu, struct hlist_node *node); ssize_t zcomp_available_show(const char *comp, char *buf); bool zcomp_available_algorithm(const char *comp); @@ -77,7 +76,7 @@ struct zcomp *zcomp_create(const char *alg, struct zcomp_params *params); void zcomp_destroy(struct zcomp *comp); struct zcomp_strm *zcomp_stream_get(struct zcomp *comp); -void zcomp_stream_put(struct zcomp *comp); +void zcomp_stream_put(struct zcomp *comp, struct zcomp_strm *strm); int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, const void *src, unsigned int *dst_len); diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index a8d22ae2a066..9ba3f8d97310 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -31,7 +31,6 @@ #include #include #include -#include #include #include @@ -1603,7 +1602,7 @@ static int read_compressed_page(struct zram *zram, struct page *page, u32 index) ret = zcomp_decompress(zram->comps[prio], zstrm, src, size, dst); kunmap_local(dst); zs_unmap_object(zram->mem_pool, handle); - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zram->comps[prio], zstrm); return ret; } @@ -1764,14 +1763,14 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) kunmap_local(mem); if (unlikely(ret)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); pr_err("Compression failed! err=%d\n", ret); zs_free(zram->mem_pool, handle); return ret; } if (comp_len >= huge_class_size) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); return write_incompressible_page(zram, page, index); } @@ -1795,7 +1794,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); atomic64_inc(&zram->stats.writestall); handle = zs_malloc(zram->mem_pool, comp_len, GFP_NOIO | __GFP_HIGHMEM | @@ -1807,7 +1806,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) } if (!zram_can_store_page(zram)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); zs_free(zram->mem_pool, handle); return -ENOMEM; } @@ -1815,7 +1814,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); memcpy(dst, zstrm->buffer, comp_len); - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); zs_unmap_object(zram->mem_pool, handle); zram_slot_write_lock(zram, index); @@ -1974,7 +1973,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, kunmap_local(src); if (ret) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zram->comps[prio], zstrm); return ret; } @@ -1984,7 +1983,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, /* Continue until we make progress */ if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zram->comps[prio], zstrm); continue; } @@ -2042,13 +2041,13 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zram->comps[prio], zstrm); return PTR_ERR((void *)handle_new); } dst = zs_map_object(zram->mem_pool, handle_new, ZS_MM_WO); memcpy(dst, zstrm->buffer, comp_len_new); - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zram->comps[prio], zstrm); zs_unmap_object(zram->mem_pool, handle_new); @@ -2796,7 +2795,6 @@ static void destroy_devices(void) zram_debugfs_destroy(); idr_destroy(&zram_index_idr); unregister_blkdev(zram_major, "zram"); - cpuhp_remove_multi_state(CPUHP_ZCOMP_PREPARE); } static int __init zram_init(void) @@ -2806,15 +2804,9 @@ static int __init zram_init(void) BUILD_BUG_ON(__NR_ZRAM_PAGEFLAGS > sizeof(zram_te.flags) * 8); - ret = cpuhp_setup_state_multi(CPUHP_ZCOMP_PREPARE, "block/zram:prepare", - zcomp_cpu_up_prepare, zcomp_cpu_dead); - if (ret < 0) - return ret; - ret = class_register(&zram_control_class); if (ret) { pr_err("Unable to register zram-control class\n"); - cpuhp_remove_multi_state(CPUHP_ZCOMP_PREPARE); return ret; } @@ -2823,7 +2815,6 @@ static int __init zram_init(void) if (zram_major <= 0) { pr_err("Unable to get major number\n"); class_unregister(&zram_control_class); - cpuhp_remove_multi_state(CPUHP_ZCOMP_PREPARE); return -EBUSY; } diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 6cc5e484547c..092ace7db8ee 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -119,7 +119,6 @@ enum cpuhp_state { CPUHP_MM_ZS_PREPARE, CPUHP_MM_ZSWP_POOL_PREPARE, CPUHP_KVM_PPC_BOOK3S_PREPARE, - CPUHP_ZCOMP_PREPARE, CPUHP_TIMERS_PREPARE, CPUHP_TMIGR_PREPARE, CPUHP_MIPS_SOC_PREPARE, From patchwork Thu Jan 30 11:10:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13954492 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A01B3C0218A for ; Thu, 30 Jan 2025 11:13:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 28FAF2800D6; Thu, 30 Jan 2025 06:13:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2158F2800D0; Thu, 30 Jan 2025 06:13:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 042592800D6; Thu, 30 Jan 2025 06:13:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D33B42800D0 for ; Thu, 30 Jan 2025 06:13:05 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C1A9FC0D14 for ; Thu, 30 Jan 2025 11:11:30 +0000 (UTC) X-FDA: 83063852340.05.866F3F9 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf10.hostedemail.com (Postfix) with ESMTP id D56ABC000D for ; Thu, 30 Jan 2025 11:11:28 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=FvP2UtAl; spf=pass (imf10.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738235488; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ti4+lyxlKNsrQ+jB6ZeK4CLAAqx7hgqRlGDCYQ/kK7M=; b=uPumfus05yGfkC8q4r2f/5CO3Z3YezQo5GxGX99CtIP+pCe15/btl8DW3oBc8E22qjdsrc Ute3jbV5HHTxddfASJ1GKJNJxf0VyalvrFlHZHbyDtnDhkf4GLb3YXGfAh+7T6fg3s87L0 ra7kKeUFrPRm78P1vWbdOiDO9B6eNsA= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=FvP2UtAl; spf=pass (imf10.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738235488; a=rsa-sha256; cv=none; b=0JEF4atvfTW/IlalB2P7WQD4dr/CBB+oINP3j3EqMpYd+J/xYvxi15Rb2EUEoB9C2NME+n Lqdz1s4x3tMXZMsSRJKJZ9L473G6IQ4cp17/BaIbBJpEzYA2B9w+RDV/PhIrdPxf0uOeMv IiArvmBUxwORmoGRHs2vYf/gWbchmlE= Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-21654fdd5daso9564355ad.1 for ; Thu, 30 Jan 2025 03:11:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738235487; x=1738840287; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ti4+lyxlKNsrQ+jB6ZeK4CLAAqx7hgqRlGDCYQ/kK7M=; b=FvP2UtAlXWWbS4a2LAZ0o0LmpmmKsWPrpHaiju72lW4jkM22HZjfGuaZiuBOH3ipib TsUN2t1coo358pRSNpEzlwlgopPfP79qIdiqUiWKHsjLTSVTZ0V3tfx7IKDwFEbGKEBm h1xguMdaUUa53SCWKcMqXbVCNjnUOB/4jP5L4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738235487; x=1738840287; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ti4+lyxlKNsrQ+jB6ZeK4CLAAqx7hgqRlGDCYQ/kK7M=; b=Sc2cY/0nwpKb+3rAJj2j1rNqchp4G/qF9hZiwmGcLAmmARbmg75xn08E5GwAMi6PuE ZVaL3NkK+ujPC+Dq5wyzvTrv1fA47MRACxbPAjjPttrSlfIg+E660bZyTgl5WoKJ/IO2 +BWjrXxXwV0Qh+PnyvsMszEWe1bdazGgg4n58O/J85q4ZZRrVZAwyWn0tzI5ir0oLyg4 zTUQD0KXXXcGOb5rfWV7hgYj38EQvb+s7cmw86h4oyy/t+rK0mfWigT06MdXpAfMUYUS KH/96vRdbZjt8L53RmEHcp+IW1FNpPTdPy1Jy45yZNaEbJ8LJtd4gP6Vc6SuMshGuxLw fi+w== X-Forwarded-Encrypted: i=1; AJvYcCXfZMNK67R0YprY9pWQFroOKWwS5+7oFSQHKGWncBpCxjUu1gkpVCyQjkInsIu08yKuNcJHmRfsQg==@kvack.org X-Gm-Message-State: AOJu0YzTCuXf1NnvzJY7+Y/XD372FKgED8Aoes1TWnWFn87xEEPP+g3s SWjdIyOAH/QdqGNuHr7eBm+oPkBmz1Tv0etctiJFq1csDB/kYZWO41pVVz2j6Q== X-Gm-Gg: ASbGncveyM9SecIntVCOJrCHWG4n63pxyQzstZl0+SzStGyqot6nenHpnvrRwHh2Yvs 7MGSXtda0a2xHfZWMQTgOzYWsnpEqWAbUPTz1Gt3CGAzROyxEgKoHFxMEwo7WLi7iJ7zTQpgJ6I vrkSeknS7APwH3HPDCX1f8r9oJmVSkmazmaq733K5eHXE9fz8jpWy3jJA5qsActskRglat5uF2+ QQ/B5oRTYOhAFa4H74ZBJY4n/5wcPDSBlhKzpCA/aDSg+gaYUOrLIZUxA4hhoO/rXZrpinDy8ne HAQHG0o5hkkskm7e X-Google-Smtp-Source: AGHT+IHPjAb1rLSQ43OqGrbcZ/IzSuiJRWw/EpDyVdzrzHNxzjb1Y6QeLs4mCWjkG7KecIPQmqzeGg== X-Received: by 2002:a05:6a21:c94:b0:1e1:a671:7122 with SMTP id adf61e73a8af0-1ed7a5b663fmr11965895637.2.1738235487646; Thu, 30 Jan 2025 03:11:27 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:d4ce:e744:f46b:4fb]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72fe631be3csm1169577b3a.7.2025.01.30.03.11.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Jan 2025 03:11:27 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv3 03/11] zram: remove crypto include Date: Thu, 30 Jan 2025 20:10:48 +0900 Message-ID: <20250130111105.2861324-4-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250130111105.2861324-1-senozhatsky@chromium.org> References: <20250130111105.2861324-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: D56ABC000D X-Stat-Signature: xp49rcc6zydprkj4xmm6a6cemg6zj14t X-HE-Tag: 1738235488-635109 X-HE-Meta: U2FsdGVkX1+7+iMVpO8lTbz8lSQ2UMYjAnuSGVih9Y0fQ70ZphYw5i5xPCI3Kkpt2MsvRbjHIpaCQwB9o/rjFYdtm9Vgcku5wKGuGkItFy8SeGLjJ3Eyy/KjW1iFwfRVdZY5OtRFpI5esXSioGWViDIZLXbMbgBuDYMzJLoskVYpd0cORDyGzr746e4lk6vJzGwIWHthvPmCZSJvOr2ZfwiBXeBGk+usfyupFuKegolagWj6NLutFEXkZVbzm38JulK6re3AuxY1APc9ot3KJscvuGCvg7te6UhJ1pFsXpg4MSsuLfJbwWvUWX2fvkWMUI/fNQs4lSixpGPYcpGI4uQvExuj8N9L2+CDaBcm4XaS4pZo8j2L43XqkqupQE1hO5ydzLZHaCC3n2O0sqSDYCDgFT2407zPJJeMcsmlmXLr6m2aK/BxPYddaBoPf7RWHYQPUZFyxqF4ZJQ7GWtYbhe+cTDWQjzpp8ptt+MydQBRrUKwXYd/97af7+PlTHMHWqIrplvr3PD6yMZIHg/NOp/qvQZ7jUcPbyt6Fhx+cTmGLRGfL7PjHMVFD28Mdpel06M9gV0964SNMKu1lxy9ggSwWLfYrw4j9hpPFKYoGhjaoWbzhxN+laB8i2Inq3YPMN/CQ718nmCm/g55pdQFcCArx3+HdMUEP6LJ2OkLXt/DHyxTbz3gjpRCYFxiMC0T9f+iAKYhsOBse38Q/3tiflzcFzVUkeipHJkZENELoupoYaLN8ddYizjCXYILWYyz0itt0r5n0d1Gidx3gds8fFnePnyg2pjG7LgJVrNlvxmz67Or5XR5AwQ0qYCeR9ULen7AXCLh8rpEzGM+9HqEdgZTe0CaBZDYiWS4facF59SGFzeJrIeEMTmS2wJXNT6h/hjXanYz2IcBncSqxK3VQ+XLY94Xp1nYJLHLh2l6Gcwr6j8dUlWZL77Tfma5auzTVsgtM6oQVrIf7OFALEz lT/7RxgN H2UDzqPywsOIIvcvxVd5hlQU6sdF8uWm9e/bh2XTbCORMlLGAYRwhyxjuHbhGxp7LyUfpk2/kuhYzI2Ny51cw+wylpPdBzXb25qOJJ3k7DDCPOpEAs0sHJ7+plfu+mXlpqXXmKPK3/g25uRJmFR3L4k+nCkYtDhVkEb6vCHNg0QB+CRmE43MogQ7orlMS0MUTiAT/EjEiHIkgeehMnIWwaHn7QkCiget39FtAKW8ZcsS3btLesxobLJuZCVbrchqIkS7FeEaRJuHFUoLCp9UXwMGHORa80c60i8mEL+CoHBntN3AMBg6P7Qj/hM6LOSHMcrhQujd2LknyyQjZrit3e2QiQg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Remove a leftover crypto header include. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 1 - drivers/block/zram/zram_drv.c | 4 +++- drivers/block/zram/zram_drv.h | 1 - 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index 982c769d5831..efd5919808d9 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -7,7 +7,6 @@ #include #include #include -#include #include #include "zcomp.h" diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 9ba3f8d97310..31b653ad6d0e 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -43,6 +43,8 @@ static DEFINE_MUTEX(zram_index_mutex); static int zram_major; static const char *default_compressor = CONFIG_ZRAM_DEF_COMP; +#define ZRAM_MAX_ALGO_NAME_SZ 64 + /* Module params (documentation at end) */ static unsigned int num_devices = 1; /* @@ -1141,7 +1143,7 @@ static int __comp_algorithm_store(struct zram *zram, u32 prio, const char *buf) size_t sz; sz = strlen(buf); - if (sz >= CRYPTO_MAX_ALG_NAME) + if (sz >= ZRAM_MAX_ALGO_NAME_SZ) return -E2BIG; compressor = kstrdup(buf, GFP_KERNEL); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 3436ddf8ab23..045032acb22f 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -16,7 +16,6 @@ #define _ZRAM_DRV_H_ #include -#include #include "zcomp.h" From patchwork Thu Jan 30 11:10:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13954484 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4714CC0218A for ; Thu, 30 Jan 2025 11:11:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC9946B00E2; Thu, 30 Jan 2025 06:11:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A2AF26B00E3; Thu, 30 Jan 2025 06:11:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8562F2800D6; Thu, 30 Jan 2025 06:11:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5DE496B00E2 for ; Thu, 30 Jan 2025 06:11:36 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D4356140BDF for ; Thu, 30 Jan 2025 11:11:35 +0000 (UTC) X-FDA: 83063852550.18.AB0DD4E Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf29.hostedemail.com (Postfix) with ESMTP id DAC2B12000E for ; Thu, 30 Jan 2025 11:11:33 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=dAORmHbS; spf=pass (imf29.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738235493; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Q/yW7cYsxX6u2D6RjpsktMUNNjtmh2H6QDs8XtL+Hec=; b=ZF86eOQljjntAH+iC31FaeiMTbDG3uOu7gCLOs7jpvof1d5aRSEW+J//MqhQvF0aSZqzhz 1RySFh1t5j8K+0fhLpHm8fwAFDG9eJFF19vYMvTuhYIz+BtpeQXytqo8nwC3p8YyvqTpOJ 3RSd6KYyMqjE27ApZ0epauTMIFnLhwo= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=dAORmHbS; spf=pass (imf29.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738235494; a=rsa-sha256; cv=none; b=JvGZLEwWF5EDgdiqFUwwdXNgtTC7Rm8SCJU8RUhU4Clrggy47Yo5UqwEgz8PBfijwikALx CutF8TJVLz/4leTLP/ri0Mq+yLsT+RwSOYmXx5PCv8QhDNzJpnQJ26R+MpzZH+NAfPv/Ag fPcd9ifBnNY84lcGWCEIbU1gLcaFijw= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-21ddb406f32so9766405ad.2 for ; Thu, 30 Jan 2025 03:11:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738235492; x=1738840292; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Q/yW7cYsxX6u2D6RjpsktMUNNjtmh2H6QDs8XtL+Hec=; b=dAORmHbS43CMRTXVC6vOR7dtCsIYct1c2cLBjFfkk385e4YKZDBwVQ0WgQcBEQJl1J h86jma0DBxJaazgnMmWIP+sQxOmvPRhbFtYY6PqHaKTdbnQTv1rFg5ZHf9qCTydw8jlH 8mPbR7aga/RRdB6yJN1VGhIMaB96Kekbflt/I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738235492; x=1738840292; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q/yW7cYsxX6u2D6RjpsktMUNNjtmh2H6QDs8XtL+Hec=; b=AWjyUxIGagoStlF70oba4yZfgCNl5aI/Pvm8e3JNF947VqufuzVpoOcnyZM9blPnh2 gjxa6jubSVkgXVb47UCPFDWuRva057ZUSTlZoI9Gctovzz8Da5dcM6fKx6/7bwAQP+h2 bMqkmFAtho3fKg2gSvmTvAchBWAdq2TVmoOmrgHAxxiQAeH9hE17s4zp4cPybP36RH2b rO96SvnnNr9Vd32R80g3U7QVafuGPNZIe9hoy8ZCbs6zJUyMATSED6q3KByCdhPH+ogJ HCbeG5Pi6aFS/oNFNe32IDWutyu7OAZEznKdlLue5Q+OV/vypOaGdSIWf8hSwQpirdCq nJBA== X-Forwarded-Encrypted: i=1; AJvYcCVTzOVPHJNAQYkpuSg2/+Q0zPlnboXhbT4/XZycnIPmFsFERyrWFUCMnK27xYvvJOAh6RPwyhQa8w==@kvack.org X-Gm-Message-State: AOJu0Ywukw72ZWrjjS2orvS8ikeNu3X87iA2NW59jRI5ssL+RCGq5g28 qguGCQhMU3E87FZ7cZ40U1KMbrutqDEwIGmJdgq7Ay0riJu/ODsg5JCq3vodPQ== X-Gm-Gg: ASbGnctjQ8nc9f4BJ2gKI3U618rwz0G2hruMtmimieJfmxSSeAiQ5nie0f/PYP5cbxf 5prJijCiLP+VZN9s62C4yhBExtREU1B/0Nk6oZTibYCZyRol340CjMO9KxC2DvbDEBmyHX93WNp q9bctuPxHf1qpMV7+dFnj2BIhDuqciDNctOu+xRD2x/uG/IBlrFun8eokBPoRjcg44a3C1UVZfQ UfKr8gFhunVYuk/YXdSGneqENIAfxr23pKftSiPWc9jUtzkZ6vS1IohS909oY81KwF32aYcozrO Hb22sYP8+99T4WFk X-Google-Smtp-Source: AGHT+IHRhQXy77fDs2DUj5z16q9LiSjBM8lSRxMlB0gLWnyP7Ox6pdNXNxWqKncG1n2YwlehavUUvA== X-Received: by 2002:a17:902:d485:b0:216:4853:4c0b with SMTP id d9443c01a7336-21dd7deef0bmr99162575ad.33.1738235492597; Thu, 30 Jan 2025 03:11:32 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:d4ce:e744:f46b:4fb]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-21de3302e63sm11331925ad.172.2025.01.30.03.11.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Jan 2025 03:11:32 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv3 04/11] zram: remove max_comp_streams device attr Date: Thu, 30 Jan 2025 20:10:49 +0900 Message-ID: <20250130111105.2861324-5-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250130111105.2861324-1-senozhatsky@chromium.org> References: <20250130111105.2861324-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: DAC2B12000E X-Stat-Signature: z4z57m6nejkxrnmscsa5rncautk4pcs1 X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1738235493-313902 X-HE-Meta: U2FsdGVkX18DSFPuYzhFFaPtF3g8Glfsi0GDUuu5lLM7AGCq3/4Z53HnfDgL08jsPSCbr0sCPkhIR1EybLfYs93lKpxk6USxyTVlXwWQY7omlw2Aq1lXriXf/cjRa3ld7AliQlxub6cKnE0CGPBu1MpY7FBqWfSVm7Pt6ummMXF8vJbPG9OYvNsgv9qDLvJm6Cxy+l8VOkTg/FGeNmL4Ev8vyoXsX/25nwIBYm3clFOTohlxs9mKev1AYWdsbyijzXLMu6fMRO7v3m69u/U7pVw0OnA+389NpERLMgpS9B5bZBHuKh0nfIQDaEjrEUSdSLOAX0glGGPJf4/sGFj7jZp+vJqJIIakY007D8j+8BfJig+ivF/RJiqVutizMjGCuWnafun6SUWtcJ90+KeStF9QrL6GjcxgYehSO9Sxwi8/l4nBt4ufBqBTd8w8UFw9dkqvBnXyLw6YWsMkaQrytWJAG2lL54QbwRGbZMVoHF7dowF5UKW/nT8TZz1rPBecFaR3iCNwQ2AhBfCgE/zYxsjjJh1hQtoZHragYTgaU6PJ75/c9IJU1a2jbXjSSrux1JmgF9KE5w0ft98CjUAgal5h5DxfaANtKnhkEo1WocbKE1RfMQ5UJEVU0/X6EfsrqOhtyPfVp1N/9aWyz6dSzClqZab+H+nSSWZUlPRpcd7iodgMbY8haUgTVqBAVy3ExzK5p9FpmqCcjNFp5aXpPgNrM0exh83/WsrrUaZ7Bnaq9JXQ0J8wJUMGIEPHYTi4BuFBHx1foG/Fm/3Cr14jm9ojSDutQuD4I8Jb/yHkXBT8WdP7cPM3+FVONAki8Pii5x331bKlFSccFexZghowe9K61223l1DXoaxrYIkvKv/SO72GqZAGkhS42IaVsB+Mvd4UJRwXV9HY/YQw2aIf/nZMLR7wY7hX7vN2gh3vSwLXnKNHiaImfm0oNLN6K+QcGtdMqJ8ALiGdtz+wHSv aDs/b2iC mGm9gbs8F5LomQSm2na37AofYNaOsRoSFz1NAcu8n9S7h+WytCRofwTJFCQedAwt36gU8pcUMmQWOvtwFlaH/Fto+Bn17LFjN6TLs79GYceb4qEzD8wjOxGz9KZmJOdHuUTUCRTYu+SbhbmpPjxxKwJ3eGajUv2Wf7J5+1orVp29SFWYf1es+PhZ1v485sVmZrfOLI8Uz6Q6hirm5GRtDNSxToF8SBjeFNjGKFmJHXPmAgGM6ESuV91F+ygy2qKx0jOhBnlh8YIEOVjrRpeH0cSzXeQNwsGnC10YRUB9QIheTIpctxTr8vZrn/uPLbvRFG9W5nA6cfUFakfKK0XJmkU/HbkvEH9zCv8dat9aSyXjuHR5100UdD7Gi1deHKQUzpZc6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: max_comp_streams device attribute has been defunct since May 2016 when zram switched to per-CPU compression streams, remove it. Signed-off-by: Sergey Senozhatsky --- Documentation/ABI/testing/sysfs-block-zram | 8 ----- Documentation/admin-guide/blockdev/zram.rst | 36 ++++++--------------- drivers/block/zram/zram_drv.c | 23 ------------- 3 files changed, 10 insertions(+), 57 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-block-zram b/Documentation/ABI/testing/sysfs-block-zram index 1ef69e0271f9..36c57de0a10a 100644 --- a/Documentation/ABI/testing/sysfs-block-zram +++ b/Documentation/ABI/testing/sysfs-block-zram @@ -22,14 +22,6 @@ Description: device. The reset operation frees all the memory associated with this device. -What: /sys/block/zram/max_comp_streams -Date: February 2014 -Contact: Sergey Senozhatsky -Description: - The max_comp_streams file is read-write and specifies the - number of backend's zcomp_strm compression streams (number of - concurrent compress operations). - What: /sys/block/zram/comp_algorithm Date: February 2014 Contact: Sergey Senozhatsky diff --git a/Documentation/admin-guide/blockdev/zram.rst b/Documentation/admin-guide/blockdev/zram.rst index 1576fb93f06c..9bdb30901a93 100644 --- a/Documentation/admin-guide/blockdev/zram.rst +++ b/Documentation/admin-guide/blockdev/zram.rst @@ -54,7 +54,7 @@ The list of possible return codes: If you use 'echo', the returned value is set by the 'echo' utility, and, in general case, something like:: - echo 3 > /sys/block/zram0/max_comp_streams + echo foo > /sys/block/zram0/comp_algorithm if [ $? -ne 0 ]; then handle_error fi @@ -73,21 +73,7 @@ This creates 4 devices: /dev/zram{0,1,2,3} num_devices parameter is optional and tells zram how many devices should be pre-created. Default: 1. -2) Set max number of compression streams -======================================== - -Regardless of the value passed to this attribute, ZRAM will always -allocate multiple compression streams - one per online CPU - thus -allowing several concurrent compression operations. The number of -allocated compression streams goes down when some of the CPUs -become offline. There is no single-compression-stream mode anymore, -unless you are running a UP system or have only 1 CPU online. - -To find out how many streams are currently available:: - - cat /sys/block/zram0/max_comp_streams - -3) Select compression algorithm +2) Select compression algorithm =============================== Using comp_algorithm device attribute one can see available and @@ -107,7 +93,7 @@ Examples:: For the time being, the `comp_algorithm` content shows only compression algorithms that are supported by zram. -4) Set compression algorithm parameters: Optional +3) Set compression algorithm parameters: Optional ================================================= Compression algorithms may support specific parameters which can be @@ -138,7 +124,7 @@ better the compression ratio, it even can take negatives values for some algorithms), for other algorithms `level` is acceleration level (the higher the value the lower the compression ratio). -5) Set Disksize +4) Set Disksize =============== Set disk size by writing the value to sysfs node 'disksize'. @@ -158,7 +144,7 @@ There is little point creating a zram of greater than twice the size of memory since we expect a 2:1 compression ratio. Note that zram uses about 0.1% of the size of the disk when not in use so a huge zram is wasteful. -6) Set memory limit: Optional +5) Set memory limit: Optional ============================= Set memory limit by writing the value to sysfs node 'mem_limit'. @@ -177,7 +163,7 @@ Examples:: # To disable memory limit echo 0 > /sys/block/zram0/mem_limit -7) Activate +6) Activate =========== :: @@ -188,7 +174,7 @@ Examples:: mkfs.ext4 /dev/zram1 mount /dev/zram1 /tmp -8) Add/remove zram devices +7) Add/remove zram devices ========================== zram provides a control interface, which enables dynamic (on-demand) device @@ -208,7 +194,7 @@ execute:: echo X > /sys/class/zram-control/hot_remove -9) Stats +8) Stats ======== Per-device statistics are exported as various nodes under /sys/block/zram/ @@ -228,8 +214,6 @@ mem_limit WO specifies the maximum amount of memory ZRAM can writeback_limit WO specifies the maximum amount of write IO zram can write out to backing device as 4KB unit writeback_limit_enable RW show and set writeback_limit feature -max_comp_streams RW the number of possible concurrent compress - operations comp_algorithm RW show and change the compression algorithm algorithm_params WO setup compression algorithm parameters compact WO trigger memory compaction @@ -310,7 +294,7 @@ a single line of text and contains the following stats separated by whitespace: Unit: 4K bytes ============== ============================================================= -10) Deactivate +9) Deactivate ============== :: @@ -318,7 +302,7 @@ a single line of text and contains the following stats separated by whitespace: swapoff /dev/zram0 umount /dev/zram1 -11) Reset +10) Reset ========= Write any positive value to 'reset' sysfs node:: diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 31b653ad6d0e..0c9b68fa8783 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1096,27 +1096,6 @@ static void zram_debugfs_register(struct zram *zram) {}; static void zram_debugfs_unregister(struct zram *zram) {}; #endif -/* - * We switched to per-cpu streams and this attr is not needed anymore. - * However, we will keep it around for some time, because: - * a) we may revert per-cpu streams in the future - * b) it's visible to user space and we need to follow our 2 years - * retirement rule; but we already have a number of 'soon to be - * altered' attrs, so max_comp_streams need to wait for the next - * layoff cycle. - */ -static ssize_t max_comp_streams_show(struct device *dev, - struct device_attribute *attr, char *buf) -{ - return scnprintf(buf, PAGE_SIZE, "%d\n", num_online_cpus()); -} - -static ssize_t max_comp_streams_store(struct device *dev, - struct device_attribute *attr, const char *buf, size_t len) -{ - return len; -} - static void comp_algorithm_set(struct zram *zram, u32 prio, const char *alg) { /* Do not free statically defined compression algorithms */ @@ -2535,7 +2514,6 @@ static DEVICE_ATTR_WO(reset); static DEVICE_ATTR_WO(mem_limit); static DEVICE_ATTR_WO(mem_used_max); static DEVICE_ATTR_WO(idle); -static DEVICE_ATTR_RW(max_comp_streams); static DEVICE_ATTR_RW(comp_algorithm); #ifdef CONFIG_ZRAM_WRITEBACK static DEVICE_ATTR_RW(backing_dev); @@ -2557,7 +2535,6 @@ static struct attribute *zram_disk_attrs[] = { &dev_attr_mem_limit.attr, &dev_attr_mem_used_max.attr, &dev_attr_idle.attr, - &dev_attr_max_comp_streams.attr, &dev_attr_comp_algorithm.attr, #ifdef CONFIG_ZRAM_WRITEBACK &dev_attr_backing_dev.attr, From patchwork Thu Jan 30 11:10:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13954493 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B7AEC02190 for ; Thu, 30 Jan 2025 11:13:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CF00A28028A; Thu, 30 Jan 2025 06:13:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C51462800D0; Thu, 30 Jan 2025 06:13:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACA6F28028A; Thu, 30 Jan 2025 06:13:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 883722800D0 for ; Thu, 30 Jan 2025 06:13:07 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 972EFC0EB0 for ; Thu, 30 Jan 2025 11:11:40 +0000 (UTC) X-FDA: 83063852760.03.BAEC7F7 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf22.hostedemail.com (Postfix) with ESMTP id B5F0CC000A for ; Thu, 30 Jan 2025 11:11:38 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=nAAKhyq5; spf=pass (imf22.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738235498; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ujanKWetHL+pQynH0HIAEO2xKM/XaqicaLx2SNCooQM=; b=nYrHMR0Fgeaa6TGtn8BZcl0HDZ+HvQrYJnzSNBaHnJ+WU0mzCDA9p1GL+idNn2aUS6txI0 4FDOHF2iPyJB7eV+KMLOD6keS84K9pC9apmvE9Vi8l58hXCiTcnFF7EcM0XJPtr419VA4A OXEmnI9tWj0uOLctTsE0lQQNhYoB8vk= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=nAAKhyq5; spf=pass (imf22.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738235498; a=rsa-sha256; cv=none; b=mjmTDevbO7BpEzr004LOXUGILxRDnr93QtBVbCtlgsrKFEFjxtLrKiskk133GgJ+VtxuBf Ll0zhvvpl3/R8vmRk5Lr9/+v+nyigpqtqz+6Hln3MGoZhP2mHuP7gE1Gi7hgdOKwCHuwCJ ZR+5XeJVUglbbcBZ8x8xfkJPC8cjbwE= Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2167141dfa1so9626125ad.1 for ; Thu, 30 Jan 2025 03:11:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738235497; x=1738840297; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ujanKWetHL+pQynH0HIAEO2xKM/XaqicaLx2SNCooQM=; b=nAAKhyq5sxN3jfp7Fp2+0s0wuGEQDqXGeQ+y5g94geg8ag6GL7Q8ja7iuC2zjMEcOe ArKGa03ZngyrEpwaAa61dtMyRqB8hrYx8Dk7NRDg0ASfsLKEzisTfMuiEuPHrW0gqW9k EPgsHpP6rdRbYUzi099wxWdtmbdeHZugKPCmI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738235497; x=1738840297; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ujanKWetHL+pQynH0HIAEO2xKM/XaqicaLx2SNCooQM=; b=r/JQJvp7ItAGZNKz5wPJCUzHCnNB8SGiLy+sxt9r5wYWudSJEihcwi54YSyoUxDH96 13hu8H/2LbHkwQXpfX6I1tY9MJqQxuozcNM6ejaPRw2+cd0WI0T51BsUqARpblNi6dND 4URJnfNRkfGELaSwYsfgeA4WycxwknoAgk+ihFp+isPLOYJwj/kOxQCfpUTyfORLxRyQ o5r5GcOcKpnXFRj0QDGAs/l0UAx2dgOyqDrB0gum+G78KoCsDiwVbT6Y2eDxZo2mM0cp OhIzwjA7lVz3tpv6hIPcMD3hJg2SVgQgNCUZy0rtVGsAag/X4pAgrwwXBsA4a+hHTkmW 2RBg== X-Forwarded-Encrypted: i=1; AJvYcCVnPAhjC/rH4BBk/w4BDj06o7ODEDDCnmsISwNVv44dFvPQ04KlLmN+29bdGCkh3EvJXvmtFk5Gwg==@kvack.org X-Gm-Message-State: AOJu0YyoAP2En7kbSoCKdMQvt9w1U+ic+j1BkETrnYva8mZOYk4YN9yI L8MYHApGwvXXYSOUJju0+X1JkOvVNysmHg0g6QZqhHUs/IsxsF/pSC95Eb4sNw== X-Gm-Gg: ASbGnctrhmPiYoIJde0aPTy79mNet+68oahFIheT7j0kPq8bNKcguyTOUQBBM1fMo2v h+96o3YlKkVcU0dIDa1Cwit3bSpdX+jErUkTdJs/xn/ve6WjEpOA1cg6RcV1lHuw5/x/uOBdFbs 1LMAdoPWxoeEej9xf5AYJ3gZMxSiZLr80Hu3E3cGEmGNxyymiVW0K5a+OAy4jLEttv/JIpB0ekx dqN+g/t8hejOlGAeOXdXkSW6RjC5CHBVhBu7aaheOc1O2yzbd7uuRyGHrPEBF0uxhyUsAe/zORd GuG/k/dVqTv+yRaI X-Google-Smtp-Source: AGHT+IGoJPBvbRFKgcHes3aZpqhjq0bpB3vf3UMoxOpT1s+vDvy+5JFl5BGgTN400Fd1/JslRst1fw== X-Received: by 2002:a17:902:c946:b0:216:14fb:d277 with SMTP id d9443c01a7336-21de196fb0amr42131665ad.22.1738235497536; Thu, 30 Jan 2025 03:11:37 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:d4ce:e744:f46b:4fb]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-21de31f836csm11410515ad.93.2025.01.30.03.11.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Jan 2025 03:11:37 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv3 05/11] zram: remove two-staged handle allocation Date: Thu, 30 Jan 2025 20:10:50 +0900 Message-ID: <20250130111105.2861324-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250130111105.2861324-1-senozhatsky@chromium.org> References: <20250130111105.2861324-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B5F0CC000A X-Stat-Signature: 9i4x8mb76hz11c5rwhcmzccec9fbp66p X-Rspam-User: X-HE-Tag: 1738235498-895611 X-HE-Meta: U2FsdGVkX19tg10KlYHCdzVfyPv04vJWI4bpTVSFc1I6ti9cgAJG7rgAL700CSPpDT0qrlyFhf5ohMM0qtuBiS1/BXn2B2/z0uePXfEkrIlu40ejI/K1mPXjll/9JgaRQZxTgJJCTu6PxUAhV8xsYUHuNs0cGxKfEgx45jXMnj4wW94sviuH0LzxB7FwVo4Xagw25Gq04KmNXBHzPBjnTP5d1oRB+rHzHVbhj1Sgga1onYemkgQ+YmWVTL9MA1AtL8H0T2CtgMgwndTfGGfB3DXnLlrd1UW7twJ1AC0Amx60IIEO/qYlFALY77XYqnaX/rawT+aJQOdhlDOCnD+pNaCus8dTUvos/4Ajj/obrrYDWSL+wnQY8HVjhsI7+q9ZKU9D/f/98d1pZubY8Zc3li6nf2jYp5q6hjQZw8DJ9k1jxBmeDoBX0bW2G7FxRUVwj+CIgoY577cfcvcj/PRWMvbxAyKu61MNpQ9MStUWh5m1V7qBDeo81f6c3Uyz594VEmzcWu+nHwoBesch86diVYoUkwom/uj4V1lnTjwEqP8rR8ghjf68/YHNvbcOu9930vtVylrxYYv6GnTJegCYJeH3x4WNCvasJ3Sjkq2muhmwjA9Px7CzFvEhaSCVHpzzSUbQML8FmcrchaEl76CuiB8DmECpk99diCvO/5CrmumBdmBf8MEDQg4RkkYci2AZHlds5yH0tteR+v0aG9R7vHl9ExoKbLNKyc9bDzOZMEmzi3GGEPSJ4g/ikYgf7BVJS+EW33K1CiVyl3bgr6z8af90XTtL1zhK2m8F2ywl/3nJ8HCvCH9JLCs89og/4MCvVqd6j9GYQyyQsOH8w+s5Q/AmOZqey/VpvKjSSQXSj3uPj17U0ijJyJsYv5df4EN4b4RN0EZ+8hHahVqzAWkWzBsBEIIjwkJsOAobxk5r6VLVp7p8ScCn6Ui6MPvUNB1j6fJZLrCA5WqpojrDM36 sp3R45Gc H69SOIIXDPJmGJw3AdtsHT3zij+MwtDNOstrpaJsZH6CkHyWB1jk/UP+u5uY2Z8YVuY4IJprepfR/wxP0KzNHYYNB9mr1PtxiqO1PtS1x+gsyG2ke0hV+sSui6mV9X2Uy7AKWWQjfkEQO1UGfSZkq+d4VvuYq6GWGSnyDRBICGUbxq4Y7xwddS7YpKezr8ak+qnr5DZgFXIzTBM+niiRWRDH4Bxbx7ObdgoVNh/nPfuiicreayt7IMSTF6E9Udl2ll7kc2dkKULi70IJIaUA5TGP0fmcws18OxYu94qNPoVx4FUdu3bXhywyemQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Previously zram write() was atomic which required us to pass __GFP_KSWAPD_RECLAIM to zsmalloc handle allocation on a fast path and attempt a slow path allocation (with recompression) when the fast path failed. Since it's not atomic anymore we can permit direct reclaim during allocation, and remove fast allocation path and, also, drop the recompression path (which should reduce CPU/battery usage). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 41 +++++------------------------------ 1 file changed, 6 insertions(+), 35 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 0c9b68fa8783..28e5b02dc3a1 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1718,11 +1718,11 @@ static int write_incompressible_page(struct zram *zram, struct page *page, static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret = 0; - unsigned long handle = -ENOMEM; - unsigned int comp_len = 0; + unsigned long handle; + unsigned int comp_len; void *dst, *mem; struct zcomp_strm *zstrm; - unsigned long element = 0; + unsigned long element; bool same_filled; /* First, free memory allocated to this slot (if any) */ @@ -1736,7 +1736,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (same_filled) return write_same_filled_page(zram, element, index); -compress_again: zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); mem = kmap_local_page(page); ret = zcomp_compress(zram->comps[ZRAM_PRIMARY_COMP], zstrm, @@ -1745,8 +1744,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (unlikely(ret)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); - pr_err("Compression failed! err=%d\n", ret); - zs_free(zram->mem_pool, handle); return ret; } @@ -1755,36 +1752,10 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return write_incompressible_page(zram, page, index); } - /* - * handle allocation has 2 paths: - * a) fast path is executed with preemption disabled (for - * per-cpu streams) and has __GFP_DIRECT_RECLAIM bit clear, - * since we can't sleep; - * b) slow path enables preemption and attempts to allocate - * the page with __GFP_DIRECT_RECLAIM bit set. we have to - * put per-cpu compression stream and, thus, to re-do - * the compression once handle is allocated. - * - * if we have a 'non-null' handle here then we are coming - * from the slow path and handle has already been allocated. - */ + handle = zs_malloc(zram->mem_pool, comp_len, + GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) - handle = zs_malloc(zram->mem_pool, comp_len, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); - atomic64_inc(&zram->stats.writestall); - handle = zs_malloc(zram->mem_pool, comp_len, - GFP_NOIO | __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) - return PTR_ERR((void *)handle); - - goto compress_again; - } + return PTR_ERR((void *)handle); if (!zram_can_store_page(zram)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); From patchwork Thu Jan 30 11:10:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13954487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19914C0218D for ; Thu, 30 Jan 2025 11:12:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A2CE7280282; Thu, 30 Jan 2025 06:12:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B6302800D6; Thu, 30 Jan 2025 06:12:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85854280282; Thu, 30 Jan 2025 06:12:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 640352800D6 for ; Thu, 30 Jan 2025 06:12:08 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7694C42C04 for ; Thu, 30 Jan 2025 11:11:45 +0000 (UTC) X-FDA: 83063852970.30.DD3A6CA Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf14.hostedemail.com (Postfix) with ESMTP id 907D7100011 for ; Thu, 30 Jan 2025 11:11:43 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=mAdAWVYR; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.54 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738235503; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5g614UtqxTvOkBYeQdwAP9Uo/j0Alc44lQaUMvbAkkk=; b=U3DisQqBkmpljGGwjMNqYqDJYO7iWZcaTV0trZ8JUoLvOtdUQbO1YJt0T+xpLl1rxg4DGS GzgxfmbbPMAkOhQOPjtW4FowHDjHN0UBww/vBjbP0EwtxybdW9mn+n8aIA25ZT6xPDP2H8 08buMJDZGbes/kC/4ozk/11wnLiV6y0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738235503; a=rsa-sha256; cv=none; b=Nae6ETYO/r0hxI/YHOeMKyPZv4WLmMQuursCtXWyZs8zOPq/X7B5P+Ob5A1+UOYXrizsxu Yt6eBSoxbwAMa20sqfGgrkpx/dm3YSaV+KPqxUqArPT1sjAK51ZxL39lsY4dK/3BMqAuF1 nLP17emVGnSyVoYydS/xfRqx230fEBA= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=mAdAWVYR; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.54 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-2ee46851b5eso754819a91.1 for ; Thu, 30 Jan 2025 03:11:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738235502; x=1738840302; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5g614UtqxTvOkBYeQdwAP9Uo/j0Alc44lQaUMvbAkkk=; b=mAdAWVYRF0L0jjsF7eggi0Znyb8Fy0G50U3EjOb/QkLoWaSdCZY/pTh7LRizx4LISw OirGfTFSIVzjfnEXJqTUms+/fpq0OSDT9N+I/Ju+p9VZXKuMt4aHsl7lEKalYaaypYrF ui0AJqDVwps8wxA6MOAOhDLOsp5pqMtr/WY44= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738235502; x=1738840302; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5g614UtqxTvOkBYeQdwAP9Uo/j0Alc44lQaUMvbAkkk=; b=XasRq1dpRwaIJ6YBjLzdF8CFdBcuHSMvCCR9XJl3exbt1ZT9QJGZeRqzQgwDiJ/Tae gfiNhM/BzHGAkSeJb6ge4cfM/dvCkihaPy2O6nVI3+YcBb2U3iFa6nxc11o8POdtaX24 CW6Ckcf+t/0jE2UyMzCni+EE2vZ6k3J+gZ3yhpAFVw9vtcUAjhVFIYouliqZTL0TH4Xb j3WZRTp9thKt1P+tWZsIj9/vmTtyWG9UtMrE1VpUZQ3+YMWu5PyU8+6TqJeT0wEPgK26 eeW81b2tnxfIezT+vvcDGIt2/dD0+FENRjFd0nWDA43nVmkVWJt0JjeIGToGoxhCkzRb ocqw== X-Forwarded-Encrypted: i=1; AJvYcCVeHeWX2RldG4SnxUXB27wP1eHmLmjBeaNFSoH6khJFQuUb1hO/ttiPtz7mEtfSmYK/GPql9d2s5Q==@kvack.org X-Gm-Message-State: AOJu0YybxF7ewDW67JgcvmN9bMesphFdwzRBhI9xbQNwY7k8wKRND6/n CM49Shax5r/XvlcgOctfIlbJWp48mAkicfqMeTDYFgZJ9GLPWbXgqdJ4q3vFeQ== X-Gm-Gg: ASbGncvufV5jYCqgv+UYZsERuEpSKDt0nbtC0QVAvBjlrRJRDAsFdHaz5Exi65x6gFW nhQ4v8Rc2fWJ39eznkie6x/3uEZaVsSDkmCkMSk8LyeD1/hyqti3hwHgr/NsM3Eal6KCa+P4Ry8 euG9rgd6Sh/jBOuyfAJ7hXdJgKVZiA3+mlGiq9Fm65i+qn8sldwqfD5OmyoFnsVOFYdACwst994 UR+siVeDUR+YBxvpUuCUyaYe8KoogOvorcZ7w19hdpqoN3AtgWUMQO+9M7oZypSSDafWr8eAaot +E3gHqKc8y2BjylU X-Google-Smtp-Source: AGHT+IHg3heFJ21F+Ok092BsmeTSf/u5V4ef9+jPuKs0cnRlT5yUCtdKYPEIddDVnItYsVqEn0xRFg== X-Received: by 2002:a17:90b:4d06:b0:2f4:49d8:e6f6 with SMTP id 98e67ed59e1d1-2f83aba9d34mr9404404a91.3.1738235502347; Thu, 30 Jan 2025 03:11:42 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:d4ce:e744:f46b:4fb]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2f848acbda4sm1333892a91.45.2025.01.30.03.11.40 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Jan 2025 03:11:42 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv3 06/11] zram: permit reclaim in zstd custom allocator Date: Thu, 30 Jan 2025 20:10:51 +0900 Message-ID: <20250130111105.2861324-7-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250130111105.2861324-1-senozhatsky@chromium.org> References: <20250130111105.2861324-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Stat-Signature: zobe6tt6jpuw3kp4fftkkuujy8xao5kq X-Rspam-User: X-Rspamd-Queue-Id: 907D7100011 X-Rspamd-Server: rspam03 X-HE-Tag: 1738235503-717894 X-HE-Meta: U2FsdGVkX196m0uFYSDbyKpuBEicjOeqvRCzAFLiU/YIXNzErPZAPiv/zd6oJmzGzuolXkabyspRKOUvOFm//frQSjB9CBd2cFH/yitgkNoJBfFZbXFPBJRw7gMb1c568siTWYj2BFAHQtyABHvwqFcXQk/x+c1DR5LAJe5rbnWMOlClC0kE784DdUJm037+jH5UOsSzS4lCBIJLrgSyu8a50p7SZFoVtiHJZZcUQsVqziE352JPNloOxBykCL2xUwVfpfeP+LDhrQY6rBrK/djT50lgKgbGzKCFNvDZoiP691h7EsgOICnShTiKEreiS0/WM2FJveJstVh3ZGYYf/OLZohINrjIZ82oh0lTNGk8zeqrcxn6SQqmPV/lxsFxqKW0Hri5EFJMu5uWxWW0j5lw/DzlgoQ5cnsPV7WaOj6A4aB9EH3wFPeokaGFfeH4HXsAgzG/vfj0Y7xdV698TpA/CIdU5yl3MjBig6r4UnJ+Dp3B4QTKHC9entndNabXpWa257lGygTfjjgcNFreOypoh1oMf8/7tAm9Z6YwZUQ5maFW5+9TXQlLNLqda04hJVYhqC+cN2hugZOG245VG7YEcn7J7LcMbJ11UQFngq3GYxOQdQ3LYF+QzuHmMGNDX/iU8noA1kj46OsqHbPzu5RMmJzzKM7Ex2xw5csbR0KRKDJnJ0UKFh9zgXOZdwhidQ54LBtgs/kN0iUwGndDlxPaCvAQKvop6U728XB4rxUSVWOy6ebw/iSvWayDWO3yA6Ho1hrhSNiM8k1QQgZ/CbjFrSV9tpVPeqGvjI+F3UbR9pdVJZ+39ftTdsn1lPrJv0HGqPskWgYPJkFoVDV7/l2rQdX+RZBsCVOQSqFRtY6LazaORXqVWPNDAmQhR2SZjXtRXMg5vP0eSUkWafwJvfH3+XddOToseUkyk/OiRIy0ndAm8rn6+AqleJieaVzy24x7i2mHxbzd4P8a4sa J0bFQPcs +qeinHzmXzwib92OVRkgamdWlMd3afVwbUd+rSeB9QiT2ZBnXTEC7tLcJ1UK5hxupzSEP9PSM2AJcwUQBNGgfuUM5Z86ns/YJMMbEIkDp0WwWu0mw9nyqmDEdb2Oj2PGE3oDWT61gYzk54wRydmiCD4DX2IMVXjDPBWFQajKXhBgP2cnubhOfMDWcCxelCpeOO0Yk23177Y3KvztDW6lHoGizS8aGW8Idusj4oHCaNnkb6HR7biXxrKhB+iV1JMZnc85HZb/b7Woj3aq5nzh+c3uFrB9jX9AHbsZ62UGvPSYOsQisxIbLRqGzQw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When configured with pre-trained compression/decompression dictionary support, zstd requires custom memory allocator, which it calls internally from compression()/decompression() routines. This was a tad problematic, because that would mean allocation from atomic context (either under entry spin-lock, or per-CPU local-lock or both). Now, with non-atomic zram write(), those limitations are relaxed and we can allow direct and indirect reclaim during allocations. The tricky part is zram read() path, which is still atomic in one particular case (read_compressed_page()), due to zsmalloc handling of object mapping. However, in zram in order to read() something one has to write() it first, and write() is when zstd allocates required internal state memory, and write() path is non-atomic. Because of this write() allocation, in theory, zstd should not call its allocator from the atomic read() path. Keep the non-preemptible branch, just in case if zstd allocates memory from read(), but WARN_ON_ONCE() if it happens. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/backend_zstd.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/drivers/block/zram/backend_zstd.c b/drivers/block/zram/backend_zstd.c index 1184c0036f44..53431251ea62 100644 --- a/drivers/block/zram/backend_zstd.c +++ b/drivers/block/zram/backend_zstd.c @@ -24,19 +24,14 @@ struct zstd_params { /* * For C/D dictionaries we need to provide zstd with zstd_custom_mem, * which zstd uses internally to allocate/free memory when needed. - * - * This means that allocator.customAlloc() can be called from zcomp_compress() - * under local-lock (per-CPU compression stream), in which case we must use - * GFP_ATOMIC. - * - * Another complication here is that we can be configured as a swap device. */ static void *zstd_custom_alloc(void *opaque, size_t size) { - if (!preemptible()) + /* Technically this should not happen */ + if (WARN_ON_ONCE(!preemptible())) return kvzalloc(size, GFP_ATOMIC); - return kvzalloc(size, __GFP_KSWAPD_RECLAIM | __GFP_NOWARN); + return kvzalloc(size, GFP_NOIO | __GFP_NOWARN); } static void zstd_custom_free(void *opaque, void *address) From patchwork Thu Jan 30 11:10:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13954485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 315FEC0218A for ; Thu, 30 Jan 2025 11:11:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BDA936B00E3; Thu, 30 Jan 2025 06:11:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B3C616B00E5; Thu, 30 Jan 2025 06:11:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9B6506B00E7; Thu, 30 Jan 2025 06:11:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 720046B00E3 for ; Thu, 30 Jan 2025 06:11:50 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 34A541A0CC8 for ; Thu, 30 Jan 2025 11:11:50 +0000 (UTC) X-FDA: 83063853180.07.C573474 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf12.hostedemail.com (Postfix) with ESMTP id 4A6AF40010 for ; Thu, 30 Jan 2025 11:11:48 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=aTHsqPi6; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf12.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.170 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738235508; a=rsa-sha256; cv=none; b=ugNJ0DkVtd59UG/2cioMEeYAcYXVHccZFUeLFGC7Ad+wKpdrDDVwI99ggZiiQP11nxAoS0 /CUjDl6y2+ubAuvztqYEAQu6ZeUFRvKujv9apex3bMymaCTBWjo7zrEY6IN11+Q+SCz/2r ycmvCk/rgkf5vyxFZ5pf7++/S+p+mNs= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=aTHsqPi6; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf12.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.170 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738235508; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wsNyTVEtqJeOeaUMXY3SZH4okPEwKCuL4AyQgcINZOs=; b=iyHE1slkLBLc2coNYKUTcUrfY0+whe7xxWKHyhSLiuTIw27fDFGfjzWlISLBsb/ckquQAI RDFLnb3zy6D4P201InOkJlRrCSScWR7CH5s0A3MzkaIa1BQqfow+Aaoyq1ueL37Vqqm1kP uvlZ5QWcJ8/4nyX7BQcqc8rtxW6rlN8= Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2164b662090so11611655ad.1 for ; Thu, 30 Jan 2025 03:11:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738235507; x=1738840307; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wsNyTVEtqJeOeaUMXY3SZH4okPEwKCuL4AyQgcINZOs=; b=aTHsqPi6U1Y+IZSxfuRyZrquQBKaJQs+/VtxvcpPW5V/J7vdo9uuShFGnf0sWuhmmO YsRyAyUD4pw1VFKlJEbZvn4GviCYcb6cuOOvAfQr9WBluRBJ0hNE3MLENTeKzHWAyejt FzYwImUEOPWRxmPqJ1gTnVDad0319CgbJTp9I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738235507; x=1738840307; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wsNyTVEtqJeOeaUMXY3SZH4okPEwKCuL4AyQgcINZOs=; b=ZIvEepgfIfwDq4Ty09v2aVzEPZn3bhNsEjIqoxfuZ/rrFrrQoCCanhgeNSE1nvVtsM o3PQ123Y1/2yGC8uOz5qvfP0k/u3cFT/HRiFKrzKoUns8NSD/GK77DE8O306kYBxCirD x/ZEHN2afhRI/ndeu4EwBAYrJde0gFF7kG3eBkGWYvfG2WvBTaz1aROpZwClZyfW+GB4 3JsxXGVWafaQLms4hWlEIyJ1mhNJLX/UBC9LMrow50o81aPO0t4dl91spr3/mzZMd7IW hcneIo2BCHNZDZRgDFSEMBWjXtogjyV4nqjs4jwLzWdQkrjRaBUaPUJ8Lj3IraD3CVKS ICHg== X-Forwarded-Encrypted: i=1; AJvYcCUZDl+L4/Da2Kt9E5UsABdHXfvED+8GznOsGlklv26EaEfx+swTF3om9cbTINvgA6jMlMelJcbPzg==@kvack.org X-Gm-Message-State: AOJu0YylM3iNWRVKyWgvwgb7F5b98tUh2thj3nTvka3HhdgBDMrPR2Lb BdUiZC2Nvx/9/MYspcYQlea1D+Qbopz8skw4gMVzB/aYxEXKx+qnBWOfTIaZBQ== X-Gm-Gg: ASbGnct64am1Q7rj8m/nh6fRNy0/8Mk+K3OKxyx0gATbgYuCU54wWByThtMFBFlk5Al 8dE/qdjqHMdeMpbabkQAeYuyy0gvmYpx/PxZ+fTEISCPJ/NgGqnCPlO4lHHG6Whr8bRErnldweA lVHY/PjfZdAk3MlRVf5TYkHtf/9cniYeLbntxSdqrjc5sTRAPWuN26cYnTlleJmdDWMFNG6Q2ZC oGFSl5DXYGlgNCSurHfQ9A3QXV5q/7/cWRkZ5e7O+6X2gjRb3LuoRtSwpqunt0QhQ7GOHXjnpc9 vnGeL3ZkQpiTG2xv X-Google-Smtp-Source: AGHT+IGkwzFoDqvhfsm5VPosd4bbEzVRTQxGnDnsn5b9Avd6d9LNbqhCO+f9O1miDGFRqxuRGkD6PA== X-Received: by 2002:a05:6a00:2448:b0:729:a31:892d with SMTP id d2e1a72fcca58-72fd0be5470mr11033309b3a.8.1738235507138; Thu, 30 Jan 2025 03:11:47 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:d4ce:e744:f46b:4fb]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72fe653fe6fsm1164789b3a.80.2025.01.30.03.11.45 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Jan 2025 03:11:46 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv3 07/11] zram: permit reclaim in recompression handle allocation Date: Thu, 30 Jan 2025 20:10:52 +0900 Message-ID: <20250130111105.2861324-8-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250130111105.2861324-1-senozhatsky@chromium.org> References: <20250130111105.2861324-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 4A6AF40010 X-Stat-Signature: 1cnyhtfuq4ap94ugywignshgcpob8gwk X-HE-Tag: 1738235508-185092 X-HE-Meta: U2FsdGVkX1/Q/Mth/IJUrYOEsf4duSOSWl0WN/T261y4LuEtaa+WOco9T9DcX3sBukJyZMpXQjEH0kXMe31mgW44SfAstxDBiIRNphyBoqFwAhm60OLDTqFj6Rvo3kv9YH8usKPX+0HW/lliZ10p4QYS0tSB1gfi477NkE9EELZiYmBhcWF+OXxBp50SdM7/HfJDJAR6XCWuGNlSP5WrdoUyqHPj8swLQrBDP0v1skR8PIm6oiMb9PESXTdoSOCrodipzmlIIMFHkPd/pNRFKFLBJbIM2sMbjAxJaKOyDfV51qyFZyj6r7ZP/eq6i61aRKoMP1YQwNstx1eyJdhTWiA8NsAEtTBDxgWewaUhwF1wcxRUnsmSK+q8ElW47Ilg60W4tBpdxyDSeRnJBR6v53m7wR3X4e4jM9YEcQZbANiLd50SgOF1xbhzXKKmqHZYT3zHMSKuZ3gY1tikMUsfkjVlUFRx4mFvOymJzgDCIbxsi9lM5vw7OMfj1+BmR8qyh/pLMucP+q//1G7UzlEhsYwdhVI8AsSaP4XLpf3RbC9Wj4nyYQKHpx+nrtsOD9vGy4iV/SEP2sjzUgApwMKf9dBIVCPSXjrQ+lDkTR6HvnlqEeWNffAi8NVXL5qI6SZAZXhG+4o3MujsDJi5o1LqsaClLu4ALnAOzUPlAswK1ocQJBVcnNiEHoufUzIdVS83cGRBVtqgNbG5aMHROykY1V65v3KtOcmL/i7j4r26uvP7Tqq7KDqGX5H/iY6aVv4NfkTPFQAtYFXMqtZZUoe+bRdEOwoJ4wkHKtpPgdKuCBsFsRpBKf2hfQGHEhpEgrMTWHkw5uSuWjWflGwmLOcyKS2RG7Ad575B9yu1buYaWrAoEcK5+SMjbvzojg2596rovH32odFOiUz3eI8tUMePdKq+gUaKxk4WJk4gyylhi85rWpl1nqJCpWHeRIkaonu/oxFCFnxTdD1LgFdwfSE 2Ki2ejcS X3oE+m59mqyaMEEyGN6Jk/94HrpJEuIfssh8Gj+eZ4XOrJVp0mO9yI3PeJXWL8xIoV537JKKKpxlhwov3dsjjBDUTab1KjznYlAQD2oOnRogE1LoJ7LMEVoeMJMDBEMkZlkvT7Zp1WO+GvgJ2Sr2u3XqORFX+sghcW1VrlVodXKw5hsk5yu6xYBkc48pEWSeNRbThkH6NQam4njXEyDFmYnZxzs1Vd3e2Wz9GYTu7F3TBBXG4N18BwMFJTSlr5ixJHq+gYF/xrcuauzuA3qvjO/Pw/9+w28N6nCOQe7ediXxWgbFlthRY9/mNig== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Recompression path can now permit direct reclaim during new zs_handle allocation, because it's not atomic anymore. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 28e5b02dc3a1..ac8b4f47a496 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1981,17 +1981,11 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, return 0; /* - * No direct reclaim (slow path) for handle allocation and no - * re-compression attempt (unlike in zram_write_bvec()) since - * we already have stored that object in zsmalloc. If we cannot - * alloc memory for recompressed object then we bail out and - * simply keep the old (existing) object in zsmalloc. + * If we cannot alloc memory for recompressed object then we bail out + * and simply keep the old (existing) object in zsmalloc. */ handle_new = zs_malloc(zram->mem_pool, comp_len_new, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zram->comps[prio], zstrm); return PTR_ERR((void *)handle_new); From patchwork Thu Jan 30 11:10:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13954489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3D62C0218A for ; Thu, 30 Jan 2025 11:12:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5253C280284; Thu, 30 Jan 2025 06:12:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4AFE32800D6; Thu, 30 Jan 2025 06:12:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3001A280284; Thu, 30 Jan 2025 06:12:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0C7CD2800D6 for ; Thu, 30 Jan 2025 06:12:28 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 38CD3C0EB3 for ; Thu, 30 Jan 2025 11:11:55 +0000 (UTC) X-FDA: 83063853390.25.99AFF5C Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf23.hostedemail.com (Postfix) with ESMTP id 42918140017 for ; Thu, 30 Jan 2025 11:11:53 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=RzaoIwfA; spf=pass (imf23.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.182 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738235513; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j8q0TE8Cfn/ZZzUdNtKaagvke7zTsDMFwn2syZsA7PY=; b=Jwct6f+/co6J1DzG7Dn+Wx4lPOTjo4rBlx0aM6F6RH7Rdrpd9s6k158HbojjU7UdRrBtFI rj+IY8CbCOox62HmgOKuARq6A5HHMTjdOOfxI+5o7/KLdrcGFziEMX8RfbCkULbyuaLnOj bgQqKPXjjdeoZX7/5fR90YnwdE/J3BM= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=RzaoIwfA; spf=pass (imf23.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.182 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738235513; a=rsa-sha256; cv=none; b=JGM2pmj9vs9nLMbog8WbYAAUdIP1mQSjrlJ1NUTvKyfyBBWODyqlUTVRBwlYOqFPzW7iV2 RjQLNEPLvwxHQM+wXinXFKP3H9x8t9VFoJGBn44CMxPpIFP5ygL/fm3/o5+quD5xwruae9 VMMB70npLqdV9YbT/cuWpr60vxzpMvk= Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-2166f1e589cso14903705ad.3 for ; Thu, 30 Jan 2025 03:11:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738235512; x=1738840312; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=j8q0TE8Cfn/ZZzUdNtKaagvke7zTsDMFwn2syZsA7PY=; b=RzaoIwfAJJntxSLsmC5gBZcowQs5WWg40tc8o2q7lvz4qIBEY5PHCRaiI/Lq0sFqa0 Aiqp/CzgyhjafrGTE810SwVEP58si+s1stdjeL2XPEUunSiiGQOkUcYsZ1itrCy1tAZv 7BU7yoHnenWfiZCUvS7Y8rKva61fSWgZCUAYg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738235512; x=1738840312; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=j8q0TE8Cfn/ZZzUdNtKaagvke7zTsDMFwn2syZsA7PY=; b=arGxECoJl+t/zLXOAuWHrJMXDVwjh9iATX+oCa89RK4Z98cnsdpdKQ0Y76XNzdhxum 7yXA9xN5cFfInGuALo8ZXvZjkqxZnH+RanCpsdEsceTLfkJTKKAn500Kj3i8bj8PEDu6 JAYyU1jMDAk44apl+ybXaJbwgKPPHqw89cwoNPCI8ztCO0Kg2hUis1PjXobvQtaTprnu oi9t/qGibq0bcKbHNQEgeq0vzUtLlYIhhoMwkMwbFBoW4lHI4bmchbdR+vfY//H9XXVM tuz+i74T5+ZOpANwvJJQtUB22BQOccYE/hQ/Rhuv741DBMFyBkAkVfeZeHSdp+edXLoi kgKQ== X-Forwarded-Encrypted: i=1; AJvYcCVaezyd06gv27KlR3G7pkiHEpoqhp4iHYeJd72H/3N7d5bSLP3NHVLdzu0H9j0zy5ot2mUnGqJCQw==@kvack.org X-Gm-Message-State: AOJu0Yz16AsBRz8kZYRe9iyN/VE9ZmkDgJBmxd2gktRN1CgJwvcFw8mD U3Ddy169MmocpOt+U1KoLDvEBZRCbMniJ/sRCYLr8jqSH3IYdB+gyw+YUFd2Ig== X-Gm-Gg: ASbGncv2Zjp8lyDl2t5Fu3Yg5Cp4jAu2zwOEoNhkL7RwF9betPRCUFpkexRyLUy6h0K Y3TvckJOP2KNkmO2QOVJ3NqWHtEpnMO1z40gjUYxnAmfaaDi608g6BcAoC/+pZbAMWjmP/jd916 GTHMOZ1k3t2GospK7Q2XdVu7FgzWwN6pBNb0oMcF9SvO74ejYkCXtL17i12tg/5EuajJ3EDdBhH 50rpk7uMMp9oRandxYmoXm29S2LmIXlsABI1nrD3Hs4sB3QPSrdNzMQ8bz5WyjGb+ZAnCRxpSOb ephPJUqc6vXQt1zb X-Google-Smtp-Source: AGHT+IERWIfj8pq2Tiil2nncQexj/06z4KXWkO06dhH+74uQD2LlR8gXztf4GGGtlqNiKmWRpbsKLg== X-Received: by 2002:a17:903:41c3:b0:215:b190:de6 with SMTP id d9443c01a7336-21dd7c49786mr81937115ad.3.1738235512078; Thu, 30 Jan 2025 03:11:52 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:d4ce:e744:f46b:4fb]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-21de31efba5sm11500615ad.6.2025.01.30.03.11.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Jan 2025 03:11:51 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv3 08/11] zram: remove writestall zram_stats member Date: Thu, 30 Jan 2025 20:10:53 +0900 Message-ID: <20250130111105.2861324-9-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250130111105.2861324-1-senozhatsky@chromium.org> References: <20250130111105.2861324-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 42918140017 X-Stat-Signature: m71qpafsf5w7i7o35xic8rasbh9ok67u X-HE-Tag: 1738235513-436188 X-HE-Meta: U2FsdGVkX1++2k5iSye5VK5W7XznpPePMqqxXYrEN9/145DiOwCvtCMzsIOiv9WTfrEGOBx+lxsAoOWKHI1EwFdmghvMYzFZ9YAOuxo/AHp5xRtYKJ8TMNVnoJULVlXYM2hVqmpGMR4+11LagT27lQA4GBbxNcUyT2L9fW2Gi0a1T4PkYsFjdsFvF9FGXO7t29E/PlPnDx8CkYLowweC+puwZp0ZR6F5wqQomd8jHnRSxr3F0CT+N1yfZauSDxTGyosVzhBABfoz7G+xO+SFPhHPLaeSpKaF8NggwMmt+rASZKkuVdgLIjQuFJ7eSxuh6hYX2TyrBo+Hk4SmYLc7Wnlx/NHkak6n+wpUJniujnqyU3141wpf7dCV4wQOtdf2omJr5Qc4bJANGeEIeFyuxfZI7xabiy1S5TQV0SI5yGYxvI1A9rjU/w4onioUgmX+kVEwVOilxA2J6UoCO85nuvBls5NPNJD14rnWRDrm1/WcQ6lUL0CpOkTbbjLf8/tSwaqU2z8HwQ0gnlVf/eXegc1SXWmtrp/XDPsSuAM11E9Ezg3T/sOO09enibdNFgbbAH8LQZJBjAIlowDJYeLvORDQxNaCSrVrSJCY96dyI6G078JR5pn73SjPvniFt1w0M2GK71liGQKaLGSYvsKrvt7O34dyb+6t/PEcaQ8D7vMyvAkGgLXeAbumcTPTYCjz/HiPUxNMSf0RcCkD33aWVF1CKfIA8cpqJQ/eZXVFmjuE/u91CODdOtt7hRVjGsPbsuoRsgI63LRHbp/cMGT557I2O/vgBjuYggN4CGHuR97fjwCVP4yvAdbM8fmv/AsAQ8Obi4UOvA6VXwcdoNmSIZ+IMf84ipbdubsjczc8EnEf/L65sOfI8DdCTKv8g2DF60p0SjkOzO6UejgZeJXIYkHJaLxK7CfI+QdYGRZ/Piq6oKtqskmyFjhfCP0jF2jqj7fNnOFTEMYDC4YQhQA IcgrO3Lz wlOtjXrv618u7DqUq53NePQf9dkF9zdp451vBXMd6qUQsFTvmYGppt8owvKu59uQC3fgWixS2Re/DRgc9sL0HRBn8hcqfJ1/u6IRd5excnLDVyfTDCxym++3WCfmAD0z6hAmKJsNeoWhvXB9k+cZtRGoERcG+APAgXUJzDAt3AcLxQnh/nKlTVfr7v6IXPMj0Z8dx9RM4Qnr9KGKW7NN668s+NET+gjZEhe0uzbiCLRzMHXzN2FKOx64jXxzyuajrcV6NShxd6zJZuOMdx6H8BUqKdOoq2qfdMED5FmWon6bv/Ig7R6KXgBRFip5qCFhOUDaCPZVCVxRTpaw3G1AMOYbgVNu46mAYrT3YdkqhjDMqnhc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There is no zsmalloc handle allocation slow path now and writestall is not possible any longer. Remove it from zram_stats. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 3 +-- drivers/block/zram/zram_drv.h | 1 - 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index ac8b4f47a496..af4bce1e5fe1 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1430,9 +1430,8 @@ static ssize_t debug_stat_show(struct device *dev, down_read(&zram->init_lock); ret = scnprintf(buf, PAGE_SIZE, - "version: %d\n%8llu %8llu\n", + "version: %d\n0 %8llu\n", version, - (u64)atomic64_read(&zram->stats.writestall), (u64)atomic64_read(&zram->stats.miss_free)); up_read(&zram->init_lock); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 045032acb22f..b7e250d6fa02 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -78,7 +78,6 @@ struct zram_stats { atomic64_t huge_pages_since; /* no. of huge pages since zram set up */ atomic64_t pages_stored; /* no. of pages currently stored */ atomic_long_t max_used_pages; /* no. of maximum pages stored */ - atomic64_t writestall; /* no. of write slow paths */ atomic64_t miss_free; /* no. of missed free */ #ifdef CONFIG_ZRAM_WRITEBACK atomic64_t bd_count; /* no. of pages in backing device */ From patchwork Thu Jan 30 11:10:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13954490 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4395DC0218D for ; Thu, 30 Jan 2025 11:12:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 79678280286; Thu, 30 Jan 2025 06:12:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 751DB2800D6; Thu, 30 Jan 2025 06:12:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5713B280286; Thu, 30 Jan 2025 06:12:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 304B22800D6 for ; Thu, 30 Jan 2025 06:12:29 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 4F52447429 for ; Thu, 30 Jan 2025 11:12:00 +0000 (UTC) X-FDA: 83063853642.09.8248084 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf29.hostedemail.com (Postfix) with ESMTP id 5CEEB12000A for ; Thu, 30 Jan 2025 11:11:58 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=AS0i2nzD; spf=pass (imf29.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.54 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738235518; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lQgITLXPodNhhg8f+6QVX/YiryNImYUIb8lIo2kz1cc=; b=OfW86AC7zktJOtc8wFplZDzZB5cenNZFPdUSEOvYOcVCpTcs1sLtsS7TqpYfqJnZYdwBrm KEqKpLPKPdkWshEsy5ziwanY24MhyjRsWoZuyoUH0Ym/enZwclNb9X/VOjsZ034/MvzcO5 OsQMJXu/KKfTIX9lyTUCgDJp4UlDuag= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=AS0i2nzD; spf=pass (imf29.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.54 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738235518; a=rsa-sha256; cv=none; b=zvJF/2BgwTM48llhUnfzuoxa8ew+i0T4KQxril66iG7n/SjWcN3s12w4BuyY65NzP+QraR tYj7OQbVavbfMWjUjpohH+q0vpKg0fyHSId4uZ7ZhJYiif4ya0YhhjG/tKjDP7I0gA2Onn bb2IK8Ev9PO4uCEDuARv84igDh6DiDw= Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-2f13acbe29bso2724950a91.1 for ; Thu, 30 Jan 2025 03:11:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738235517; x=1738840317; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lQgITLXPodNhhg8f+6QVX/YiryNImYUIb8lIo2kz1cc=; b=AS0i2nzDCD05U16ft9F1Ey8Vl5TgyC0hm068mn1mDCbFpo7sVihUjPDGcakhezw1pZ at6Tv7Qsy1RQAyXwijIWCAhYC6EM3qBGJjpviDgakjPPu4qB28ivsH5tho4zDM5dZ2WI EGUqwZWCIXgT2NrJNJqrvAtFg/V8yzI5jlZKc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738235517; x=1738840317; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lQgITLXPodNhhg8f+6QVX/YiryNImYUIb8lIo2kz1cc=; b=dAnukYthDohdVmAh9IFwaN6Ngl8yfzPqPoszRppdZXh+P3PEI8qibGop9o28Pv7TJV X48tuzJn5tQ0LvKCHJSnM/eeAjr6/LHr74EVyixkNp7BsYRh8zlqgc7gq237XoWYuH0k jz3QXMZvbfZhrbJS6SUnVz4yUWq8++Hn6o5O3E9bPBLceeoGeDsWkjcr6yKFnZkkcbVB A2EDP7zSlZHaCJEhxkCTdvBbRJD+N8qJ2QfUC0rpzW9bh9GWBaMiJo4/9BmKE5gK3Af/ RlJ+Lla91HrjrowlHbZcqA2t8CNo9YQDQlWAWltFWUauNiM+SRAo0cPIzMpNgq8KgXtV K8UQ== X-Forwarded-Encrypted: i=1; AJvYcCVGs3eHPZCBrTVkGhjXIFD0PPpBU+WyQE9MhAKUktULuKZ4OoouozMKLE3dzOL/U106M/qoJJSggQ==@kvack.org X-Gm-Message-State: AOJu0YyXNmS6rkMeRg2PoLKEzhT6sdNlyOIQ6mS3beSvFXbS2rWsyC28 8DeISskwxqEMRTSBG9ot83aNzuR6f8Fx9+zLKPYaokEgfDnoOCxfuov5je+nuw== X-Gm-Gg: ASbGncuXPKjKIhm0ofh7t+hZhYlziLcpdv4MxOMJG2kTTnn3HYBzS1gCUdG1/p6mB94 s3Ngbzq7hzx7SfpDHqLM+8bGMp+iKhkVUfAGx5ooCIyJvqIGsCaVEPZ++PAXcjDR1SEEvAD8PzR RtFEjkXSqO1kPEgKgN6rhd/0y957LLDdkYdv9RvdW/O01JQbehX8gb3qZiGMcnUr2OST5qMBkYW CG8eBAsYHHqIT9JuqNpQ+giFnBPBiWIHnOpo5AREhX5qzJSoAP/y3hLm/gCit+aIGtNm2wq+1rq 1JRjxfegeLIgZGwg X-Google-Smtp-Source: AGHT+IHdIqAq1Ip7rjsfMeTr4P2qYVZikb+j/nc93lXmG1/0trH7vosRiIP+YWEpphAlbCSobUlWtw== X-Received: by 2002:aa7:86d6:0:b0:729:425:fde4 with SMTP id d2e1a72fcca58-72fe2dbbf37mr4157189b3a.11.1738235517164; Thu, 30 Jan 2025 03:11:57 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:d4ce:e744:f46b:4fb]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72fe69bba21sm1144810b3a.114.2025.01.30.03.11.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Jan 2025 03:11:56 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv3 09/11] zram: limit max recompress prio to num_active_comps Date: Thu, 30 Jan 2025 20:10:54 +0900 Message-ID: <20250130111105.2861324-10-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250130111105.2861324-1-senozhatsky@chromium.org> References: <20250130111105.2861324-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5CEEB12000A X-Stat-Signature: t1niqwmwfaicfuskub4o95nqohk8d83i X-Rspam-User: X-HE-Tag: 1738235518-403572 X-HE-Meta: U2FsdGVkX1+Gsnqg+yv6aC+hmQXfoGsL2T7jwX7+r303gxzvly4y3KrvRk/sMVhO/hnQhUBaLpGjMlYfFOY3ndh+/Vyax6j8fQOifJUqefXZsFEIu7bpMep/RuGRMT1VyeTkn9m/gBeyvuS0+u1L6+hLKsPg+0e+/pfNd7DWRiSkZjxmVQeeITrQY+UTT0v1DaEVU3TjGO60XRByXsoye4czhKzEvHt1KWntFaDCvt5F/EBO8dp5gTfU8tXiifrV/6KJatFTcAkK1rnd1gL+d4ensRROGok1H333U+qGOma5HiRzEAnh4diX/9zb2fEiU15izrhmpXjACSwm3+Bc3ZrAL9nxacK6Nfe/x5DhjRNQmjw+Gtr4/02VOzARW/5YrO1X6L/pCKjWhFhy6vvwcTZxfqj3kvs5xCT2OQLZzIZI26iPPIBCkA15T9MQV4oL7r5AZUQZYtpxnysnAeNhb1V2bLgUPjylPW4Cyvh1CmiUAaAqzj5pu2+AIVNXCg/pBkyl31k+Gaxl6XVvRwqR7ep9yuahjYIZvQaGyoer6tn2Am8pcmNFiYKKF2vbrBY2wZi8iXCulYL4uGZv16CenjMd01bzPnb89ZBCZTPOtl6DWrHBbDSlcMUzhsnWR44IR9C4RJbZO5ds7j9tga5J4XYzhK3NUP2U5sSVnEFW/6cYMvx06B+m8dBxKTQtzFkJB++efM5Mroubwkt5PT4AXF2EXCA3/VDmN4VpgmTmaj2X+FqGdxrGDjCGxvAEY4pCr1q+dMDeB5VXXAs83L/ZwZD7G8r75Y1rZ0fu2CULYcKTukB9vEpnIkBrOSGuZJyk/IPyA2v3cp/rL0hdnP8WB9OY3wqUx0GE3MJAXvEVUZT5PI+G5tTwUf0plxeaLtK4181iQi4kwuWf1Nnk8hPti4lm9A1iiNRJl7wu+Xk99n5pv1ZdAiwrKUjOpkDMHs8he5sEcMRlkz6noxTq71N Ftqlw2KG yGQL4AsjeYRN56OeAJzE2bBmSi1ikNMwx3tJ2v1/zd2k+/kl6nfC3Omh2F7jdLHnKUGp7yz+Oni43sYKyM9fNwVPocjgknw2yDCG3Hwy/qnQB+2CbW3wcGNmDgexW+vvWiQywnjUQxC/E1MiCcvD8P7yrm6aMRI6QEZ8xWOnbICVsftzlDgtJeD8sQtrX7ag9dh407o6kICXuHAwozFxhGcrAe4AYpe5Az3IIrCIJtUBqkP0B6CQ3XV7Ot0TKxftzI8ngyRCCZifPHgrUv8OMTLE3sM5S2MLBOhIBB0hlM2ybD3XisRwhZPFb+2E0NXakxJqVuu//ZdqW1FmSdcc/NM7aVXAoys1SS0z9+maFMX2uihU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use the actual number of algorithms zram was configure with instead of theoretical limit of ZRAM_MAX_COMPS. Also make sure that min prio is not above max prio. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index af4bce1e5fe1..3e91fd1dcfc7 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -2011,16 +2011,19 @@ static ssize_t recompress_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t len) { - u32 prio = ZRAM_SECONDARY_COMP, prio_max = ZRAM_MAX_COMPS; struct zram *zram = dev_to_zram(dev); char *args, *param, *val, *algo = NULL; u64 num_recomp_pages = ULLONG_MAX; struct zram_pp_ctl *ctl = NULL; struct zram_pp_slot *pps; u32 mode = 0, threshold = 0; + u32 prio, prio_max; struct page *page; ssize_t ret; + prio = ZRAM_SECONDARY_COMP; + prio_max = zram->num_active_comps; + args = skip_spaces(buf); while (*args) { args = next_arg(args, ¶m, &val); @@ -2073,7 +2076,7 @@ static ssize_t recompress_store(struct device *dev, if (prio == ZRAM_PRIMARY_COMP) prio = ZRAM_SECONDARY_COMP; - prio_max = min(prio + 1, ZRAM_MAX_COMPS); + prio_max = prio + 1; continue; } } @@ -2087,6 +2090,12 @@ static ssize_t recompress_store(struct device *dev, goto release_init_lock; } + prio_max = min(prio_max, (u32)zram->num_active_comps); + if (prio >= prio_max) { + ret = -EINVAL; + goto release_init_lock; + } + /* Do not permit concurrent post-processing actions. */ if (atomic_xchg(&zram->pp_in_progress, 1)) { up_read(&zram->init_lock); @@ -2101,7 +2110,7 @@ static ssize_t recompress_store(struct device *dev, continue; if (!strcmp(zram->comp_algs[prio], algo)) { - prio_max = min(prio + 1, ZRAM_MAX_COMPS); + prio_max = prio + 1; found = true; break; } From patchwork Thu Jan 30 11:10:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13954488 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6C44C0218A for ; Thu, 30 Jan 2025 11:12:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 572BE280283; Thu, 30 Jan 2025 06:12:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4FB532800D6; Thu, 30 Jan 2025 06:12:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 35257280283; Thu, 30 Jan 2025 06:12:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 854432800D6 for ; Thu, 30 Jan 2025 06:12:22 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id F1C28C0FAC for ; Thu, 30 Jan 2025 11:12:04 +0000 (UTC) X-FDA: 83063853810.01.9ABF556 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf30.hostedemail.com (Postfix) with ESMTP id 23C2380009 for ; Thu, 30 Jan 2025 11:12:02 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=b+yu9dfD; spf=pass (imf30.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738235523; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=M1+IQlt9pyJwpRT7WcoMT+zWDZRWfxL4uFL06pInV4g=; b=5Zbiu7F45jGknCuylHPbtgCwbLwS5Z0pkF7TCDMEZCG2OC7v7ptrVUMiDZV65VwWV9HFqx +ywhtlEWyLwgW+49NXTaLukdDy1acF45602w21BVobPP4rkjO5PboG9exNxq9BgpCc1sUM doZ5ihpuVA05+ivWsrnzBTUlR97RJrE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738235523; a=rsa-sha256; cv=none; b=JOkOPhB18uzkznWoJ+PMrR0/TO1j2CTopI/AsVWj6pi8wnfCJAEIJp1eg+oPIg4SVRjOP8 hHc17fFMCtcW2Gb4Y8ixc4xg1Xg01qYtl6x7AotnZlMvm96hl8zX8T+eqTtxJzJHMB/drw OxtU6ut1DwX8SXD59m/K3dCRM3X2iPQ= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=b+yu9dfD; spf=pass (imf30.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-216395e151bso7203225ad.0 for ; Thu, 30 Jan 2025 03:12:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738235522; x=1738840322; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=M1+IQlt9pyJwpRT7WcoMT+zWDZRWfxL4uFL06pInV4g=; b=b+yu9dfDradCU/rMeJ4teqxLO5Sa10P5C4tBQkzGD7cZ/ys+RfkjwYMnq8dDIarS/P YqiDevI8q0Sz8UgAjqTF2XThmQYR/fQkrIKLEveSZd/m0Xk65kROE4fallnLI5xdncIg IMZmvjfG3ZwGxotJZcpDfXe51AWn1CW6M20js= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738235522; x=1738840322; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=M1+IQlt9pyJwpRT7WcoMT+zWDZRWfxL4uFL06pInV4g=; b=BnsoKsysQiT+X9jwff7zxTICJF7gsx2zVO+6+S+5CXvRbfAAviq+swf5q5YiVI2ntI JalqgC4QrEygbpk8O7Qf5nXzorTz4ws8JQRMqEGuhuvHaJgCEWm+7xQq2zxqLuF+O4/i fLFLroaZssSU2SVNOn45lMMF6g86O5sZXffJyLM4DO/s7dlE9qWRqXr1V3uIgr04oZJ+ laN2BJGkeKJCQLE88jZrhmzR8Z5CR+x+GIO0NOenmLpuDSkUNYy4Keb6T5SNFTmglhga vJi0jmPiGdrbLHRPSCgCbYIxppl4vUJyRhCYsf7oUQQZoGGN0tMwSpujGkfL1evZVq9n QBvQ== X-Forwarded-Encrypted: i=1; AJvYcCWua5soWBFxsND7IFQvzAn+CPbRgXp3NTEPb2pVZtIq2W+psMUZXQo8fZ55Lm+IIWEjPt7DbTvjgQ==@kvack.org X-Gm-Message-State: AOJu0Ywc+ditxY7xj7y91H/JemeJ+X07Wr2uqv1Y9Sgre15ALwuXJybu BVB/1YsDnuD+DrO+sfRjYwJpLg6yCWs52Cqvnzl2Ep8BmKCFMeTG66wvFBZOhQ== X-Gm-Gg: ASbGncvRsB6TfBz6bFKVZChrca1s6mE+4hf6PAZt9B9QUmLra1O9v5FASuky71kmC9G 5qqqnX3/khalJbIVHL2v8uy5GZb+Ru37hl3s14fZPKokSvR+exCy4icwjbPKffC2U8bTDNSJB1Q Y+ucjOm9+ah4UW1NDcNJpMDBHjfag1tUcQvyoMfhiLDSlz/KgGT8QzZ/y+TTuSsrNV6Sw0eTOpi 6+V5g3q0nxZ41Srd/ZiCUOf9JaYhI+cKsp0UYHyzhVNslqKykWEoAGs4TlavaJnkg4zRVCiODtd DiUXkSvjlQzD0lkz X-Google-Smtp-Source: AGHT+IEpCpAng/2jtEa7zxIDkv9KC8ilU6auGAeJO2psWZMa0QEO4BHwp4dFoq3oCn6eJDAqgONEiw== X-Received: by 2002:a17:903:2284:b0:215:a808:61cf with SMTP id d9443c01a7336-21de19ca626mr48669045ad.25.1738235521983; Thu, 30 Jan 2025 03:12:01 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:d4ce:e744:f46b:4fb]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-21de3300880sm11526865ad.168.2025.01.30.03.12.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Jan 2025 03:12:01 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv3 10/11] zram: filter out recomp targets based on priority Date: Thu, 30 Jan 2025 20:10:55 +0900 Message-ID: <20250130111105.2861324-11-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250130111105.2861324-1-senozhatsky@chromium.org> References: <20250130111105.2861324-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Stat-Signature: rsbd9xqcnmg8brmbdba9ww6puuqtgey4 X-Rspamd-Queue-Id: 23C2380009 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1738235522-536786 X-HE-Meta: U2FsdGVkX19B/1Aa07x4ahJ//Kj2z8JIIONZwww2TuiRAvCwQjsvdxGpH2GOgDfVUt3jUyWZHvCWnEhZUKV0/a4faR/JB/HaUU3GFr8nPPnFjhrEuNoNwkVkQhMRmdjF3PShKlJ8s/oMjkg//4v2wXXswMo3Tcq6bFT8/W92CUkgLvLy7wkoFQ0JGdmFMFKmxCLVpYNJp8whiHjJaUMQOvN9mK0JksAdMy6RpqC6ItrzpGBcPl7rCxcEfAXCqVbTIGwRpZxKu2bJn5kO7j2pc9FgIw33SQUMak++MJh2BFm6uuWsKtTLnA4ZMDcZEZ7mehU6UqHcnqAc5vjch8nxhSKtTsHzgjc/xxNt1PfX9nndCJQmlL3MRqJhZdZ+5e+om0UNybOqrpKcAfyW4HUlBlJKij+T5PodCKlq8NVyfup4nMZUaCQVFjKBjLIrqyhQ20G51XBlZry3PnbDCYKu7VRQr13rAXK0tLm5mCdqwgXiGUwYPMbxoQoDGF1e8gvlWlftBIjJhHNK3QP2ebgywfUY+jmWRJEGFZv6i8+29uimXBUeOKvOK1wssu8FvfaLSZHEMSJUThpDGQzvpXmkGUERQ01XleiO7MDx8VUnRCka4Qu4FK+vylFzAlsHM8zI6OLnyv2H/R3oN+xXUOV6PfX/501+3/e3efF2PfPJEVoQL3riDTABNoBW2AzcQaxrUuRpQgIJsQkZYusmLbZG0UUHKl0bSKO5fDXlR1Xs17WGP+zXQOGLWUzeOj+oy1jbpD6ByJ0Bk2OhRAc7PaOU3ifnPgr/fur75TcVCKvGw3O0CWxXWxg0OdE2jX0Q74CUITbtwtAPfzQvV+BNIrs2X5vwswxfXhW+J2UcLgzB50NPMaY5Du8vVUrpiwD7teDSyS0HFlRyHEWgxE2xSvLj9+fI48FqXuQQjg+uSJ+EkHS7/o1lh1HJoBSuku9JaIUuebMX9mKP2lqbBFBJ3li MlmwsseE hN/2rzjFX0Q6R7V/mFSZKvb8qu/rbglG/q8C3rfH7RjW4CiVgiQeuUfZLZC1qzAL4gUOzZOx9XgneppqWL7786g55ArwQOMRVRckbTocyvHW6ZAZdFKR8oME/S5I98oHt4GqhFPaL3xYYxl9sx4KY6OryBXTUKI6ww4nP1fuWV9/FYI67RaQvkQrapWx1/X2+hKs0+dD98sfeQFXucrn8S0PlSNmJPjKbdaSgNKPa1xZsB9HtvA3j9Gtg7EnreQT/MBa80Sv3wSQkQ6OpBC4FLzhmRdZe7j8PoOdG3LefFASlN+cLwwzhznOzIg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Do no select for post processing slots that are already compressed with same or higher priority compression algorithm. This should save some memory, as previously we would still put those entries into corresponding post-processing buckets and filter them out later in recompress_slot(). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 3e91fd1dcfc7..500d6c8b17fc 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1813,7 +1813,7 @@ static int zram_bvec_write(struct zram *zram, struct bio_vec *bvec, #define RECOMPRESS_IDLE (1 << 0) #define RECOMPRESS_HUGE (1 << 1) -static int scan_slots_for_recompress(struct zram *zram, u32 mode, +static int scan_slots_for_recompress(struct zram *zram, u32 mode, u32 prio_max, struct zram_pp_ctl *ctl) { unsigned long nr_pages = zram->disksize >> PAGE_SHIFT; @@ -1845,6 +1845,10 @@ static int scan_slots_for_recompress(struct zram *zram, u32 mode, zram_test_flag(zram, index, ZRAM_INCOMPRESSIBLE)) goto next; + /* Already compressed with same of higher priority */ + if (zram_get_priority(zram, index) + 1 >= prio_max) + goto next; + pps->index = index; place_pp_slot(zram, ctl, pps); pps = NULL; @@ -2134,7 +2138,7 @@ static ssize_t recompress_store(struct device *dev, goto release_init_lock; } - scan_slots_for_recompress(zram, mode, ctl); + scan_slots_for_recompress(zram, mode, prio_max, ctl); ret = len; while ((pps = select_pp_slot(ctl))) { From patchwork Thu Jan 30 11:10:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13954491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5BC1C0218D for ; Thu, 30 Jan 2025 11:12:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4DAF4280288; Thu, 30 Jan 2025 06:12:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 439B22800D6; Thu, 30 Jan 2025 06:12:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B2D3280288; Thu, 30 Jan 2025 06:12:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 07E1D2800D6 for ; Thu, 30 Jan 2025 06:12:31 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2297147095 for ; Thu, 30 Jan 2025 11:12:10 +0000 (UTC) X-FDA: 83063854020.28.B1237D1 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf14.hostedemail.com (Postfix) with ESMTP id 3319210000E for ; Thu, 30 Jan 2025 11:12:08 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=CXYuKn10; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738235528; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MLxgJRPaVNwNF6Kcm4INbLog064v7nke9leBvmEg3e4=; b=M7Eb6V4ctAd1sLxWyV2/T3uIid5PpuyEI9/Q++aB50Lv58brJVAS7v8kvmreUc9PZNMdmm RkX0iTrqpJGH566fVj/hilKq8wFsliJYrXm0BtzGLViyzCfGPmZ1WRCmVyhtfGXa6eXEkM QjUrIzvTT1o7wqXfov4q2H9H66mPwnc= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=CXYuKn10; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738235528; a=rsa-sha256; cv=none; b=gSa15OVjL/NuLV152p9lsPqDcWF41NL/DsH9g20gekkb97422v0NXX/JfEsJtPkPYoCbP5 cMWhoL6eqAgRNYqY3MbKSHp37BKYYmAZ5Ktjg2EQ3FYiF4bFFZt4FnD2T7G7ADZklcBWSc kSfFg7hdhXK1B4UwH3dWg6NyZwZjxnw= Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2161eb94cceso6961845ad.2 for ; Thu, 30 Jan 2025 03:12:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738235527; x=1738840327; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MLxgJRPaVNwNF6Kcm4INbLog064v7nke9leBvmEg3e4=; b=CXYuKn10dbv58WqVoL7XTwWDWI0HLqY3yDIR3J1pchqVpytzS4ojatJlxx/lTeaZEz 8tGcfDrgWlJeSviwa6fpkZMXQKy3LzTLHpTA2sRqMkqlAgFiIDAhpUf+sfWq9Hn2O5rX RzyiWIOwIKN6ZECkjNiOSRXkTQCNrZhCvVxPs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738235527; x=1738840327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MLxgJRPaVNwNF6Kcm4INbLog064v7nke9leBvmEg3e4=; b=TuFSLRe65WH0B0OMPHsKyFiCh4jwCbok/DQPFKF2VovvW2cBNperPN/VK3id7KSy1L TgLFdPZrYpYXzOyQ60UJuvV5Ge+PbT429Ntqh0QDO9tHF5FZCjMaTT72Zfl5/AIvj9tg YFNPHayn+kKxpg1j5H4U/qT6+b288CKAJpfKGY/VmNtnfWYrp+3jHkSmSzqQr0qMkTA4 v4JDrDb3juhnODqA/2gCCFmouEZqiM9okS5bbUQ2sTyodgfDpGwZ1jy7aqKCTUipD/dG MZRKw2F43e1i7LJSY8/bzZ0LFquaM1rlJJ1n9i0yRSehl1yKfJsG+zMtiNozG+6w9Gsy C8+w== X-Forwarded-Encrypted: i=1; AJvYcCVHHlimJ+o3JZ9BjZlyU4cITHYUnlocwWT7Gf8JZa1B21UCqK6+5jikkXIimcYPgsAG+p9pt+aXKQ==@kvack.org X-Gm-Message-State: AOJu0YwIhbz+GnoXFrVSTZq0/PJZgsiBgHCZt0oYIiwiHD3SzgJH7D/q C2yOnY9IKyc3/KyWgwlCEym1zbzyK329mPROhV1LhgdozKIaNt3bmBLfrsZw+s2WHj7xL64zu5A = X-Gm-Gg: ASbGncswYwdelbLKKGrQMDot1Ph9CGdC+cW+a7L2UhyGdCx95YCQJNEbNAVZpzIC8Gr nVmvtO+oc9ihT9ww/cMY8ArDn/9gEcIeNwloIQMZe4d5jqwI7pVL+xsrZSthlxPUv89eZqOc3Pz /QcNQg67vDWoDPc1fPuGOnocJLCDbBrZNC+j4cbx5VSh8L8y2zINvTj8DWAm2mx0al+hpR7PuhD wHv/mNJQYBkeR/LWEuRPGQXt/nZY90jnCjPDB3rOhNCAQmew2CcdVrn+lAeJvqUT1CtnNntKIF8 XsmY9CpI/X+SR44z X-Google-Smtp-Source: AGHT+IEmgqIzatKzhSxji3bLavuavRKHjn6PnEoK7vJ9Qf8RpneZSWnCVW2omFdSy80mZ0nOsxnoUg== X-Received: by 2002:a05:6a20:9c8e:b0:1e6:8f39:d607 with SMTP id adf61e73a8af0-1ed7a6b17bcmr12484502637.31.1738235527034; Thu, 30 Jan 2025 03:12:07 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:d4ce:e744:f46b:4fb]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72fe6429218sm1210151b3a.66.2025.01.30.03.12.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Jan 2025 03:12:06 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv3 11/11] zram: unlock slot during recompression Date: Thu, 30 Jan 2025 20:10:56 +0900 Message-ID: <20250130111105.2861324-12-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250130111105.2861324-1-senozhatsky@chromium.org> References: <20250130111105.2861324-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 3319210000E X-Stat-Signature: sy16n5pdsuqpytwb9wjwqnbhdtt4ndsp X-Rspam-User: X-HE-Tag: 1738235527-61490 X-HE-Meta: U2FsdGVkX18AmIvfPDFafucSNuEM8yi3m/kk4DbDgSVdj85V4HtoB3JUIPk4bOu2TcU0r0VvfV1GZQsgn2ICqXKnb8B+SUG/qS9hKvGwHJF+8eug6x4wyFdYWc1s3vMTrABxHAXkRkafDr/WoZZUYgxwTaHcWz1k8RqDjRMHryg3GNJ1ABdSmnvl/men1nBBqH1oRjqIt9MoSBOXN9JyP66fyNz9xwv5jV/PzzdZ+i1TH7YG28HrcPjZt6Df7My5+AgJnCYmn5Lzrh3uA5j/udLpE43OByOBmgfoms4djDJ7JD/nTK3tqTAldIWSQ/novxD49fMuHWLaYKGuNxZh1Z6UjaDVzhl4JgtDiXD3x07e/o6vm/s5dm25xQzZ1d3qPUYTIoQzxejhx19KzsL711x8/KxgItKXNtEQdK4DF+0gNgWQp6XS5ev2NdCAR3PseQg2kf+6rEhqbtNuFlQ1GqMLfdNyAPYtkF6KvBHUAI5n8mIYR5yTLrIK92u2/7mAqDmvjA3eRBcx+BXqz8ajiUOwSdSCkR9TZuPKhwwo+xuzgtBBVfQ16PyJmV1R49vEgOZatV67P4bTANLCd4Z5cuAS3+1ghbu+rSXVEkAr6xkI3IxEMuAmDFAIAwpuaBUi22lsBBlCF0MjqNVUfmM6tp40butOuW5N4LMKNkxmJAYrMnbJOYdgnkjvgWDZyZ1+m3pzjpLVIkbiVLLhJFj03RrJCRhtdhLPFYtLzwque+LwMQFKDlmii9gAl2cyi+7clSLxJSkaXt5HSBXz3ZTNx/2+JILPzyp7jH2zDhgCmGlBDgVaUSid6XjDcDrQoDhO69JX0/VkFDnKG4k2rdBDc/1/jP6nkJU0SsDMi51F1gG7wjHC2kvfbDW/qT8z7ENiZ7ZxgOyF5sgWbYI5f/JDIDD7Plm+ybCq+ZDaYlAk8pqDvGI1v9FQycByf57cPUO1GS6neQ9mhUSux7r245s lANX673F kz0+Ujy62BMB74tROFfNNEx78lAmuHZ6jilDQjtqdHtYNV+gXJk2g6J//T+7Wt3G1K4MMfNrwkeBnTmz0D370x9vM+cDbo8oRHdKT7NzbcrbbqMRFv83/JY/n7j7modPazyB5jIJHktOdNCkpoEVxf52mmPtuRNZStUjFEqxom+uKOvcSeLUKNv2v3YSss8CR7fro/pxHLtY+cNpBD8/db2mx5TAhBnZbVOpUX1bgj5RC/FBIiy0/UaUJ8ClQeIBn8po5Xxi33o4dex4ETYcmOJMcKBhZJlSVnmwbAggIVvvRwd5fHv0oXlb7q+shYX21jJb7qeaFFfDVYBDOWSluNtwc9g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Recompression, like writeback, makes a local copy of slot data (we need to decompress it anyway) before post-processing so we can unlock slot-entry once we have that local copy. Unlock the entry write-lock before recompression loop (secondary algorithms can be tried out one by one, in order of priority) and re-acquire it right after the loop. There is one more potentially costly operation recompress_slot() does - new zs_handle allocation, which can schedule(). Release the slot-entry write-lock before zsmalloc allocation and grab it again after the allocation. In both cases, once the slot-lock is re-acquired we examine slot's ZRAM_PP_SLOT flag to make sure that the slot has not been modified by a concurrent operation. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 80 +++++++++++++++++++---------------- 1 file changed, 44 insertions(+), 36 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 500d6c8b17fc..a6bc1c2dfbe6 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1871,14 +1871,13 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, u64 *num_recomp_pages, u32 threshold, u32 prio, u32 prio_max) { - struct zcomp_strm *zstrm = NULL; + struct zcomp_strm *zstrm; unsigned long handle_old; unsigned long handle_new; unsigned int comp_len_old; unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; - u32 num_recomps = 0; void *src, *dst; int ret; @@ -1905,6 +1904,13 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, zram_clear_flag(zram, index, ZRAM_IDLE); class_index_old = zs_lookup_class_index(zram->mem_pool, comp_len_old); + prio = max(prio, zram_get_priority(zram, index) + 1); + /* Slot data copied out - unlock its bucket */ + zram_slot_write_unlock(zram, index); + /* Recompression slots scan takes care of this, but just in case */ + if (prio >= prio_max) + return 0; + /* * Iterate the secondary comp algorithms list (in order of priority) * and try to recompress the page. @@ -1913,24 +1919,14 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (!zram->comps[prio]) continue; - /* - * Skip if the object is already re-compressed with a higher - * priority algorithm (or same algorithm). - */ - if (prio <= zram_get_priority(zram, index)) - continue; - - num_recomps++; zstrm = zcomp_stream_get(zram->comps[prio]); src = kmap_local_page(page); ret = zcomp_compress(zram->comps[prio], zstrm, src, &comp_len_new); kunmap_local(src); - if (ret) { - zcomp_stream_put(zram->comps[prio], zstrm); - return ret; - } + if (ret) + break; class_index_new = zs_lookup_class_index(zram->mem_pool, comp_len_new); @@ -1939,6 +1935,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { zcomp_stream_put(zram->comps[prio], zstrm); + zstrm = NULL; continue; } @@ -1946,14 +1943,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, break; } - /* - * We did not try to recompress, e.g. when we have only one - * secondary algorithm and the page is already recompressed - * using that algorithm - */ - if (!zstrm) - return 0; - + zram_slot_write_lock(zram, index); /* * Decrement the limit (if set) on pages we can recompress, even * when current recompression was unsuccessful or did not compress @@ -1963,37 +1953,55 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (*num_recomp_pages) *num_recomp_pages -= 1; - if (class_index_new >= class_index_old) { + /* Compression error */ + if (ret) { + zcomp_stream_put(zram->comps[prio], zstrm); + return ret; + } + + if (!zstrm) { /* * Secondary algorithms failed to re-compress the page - * in a way that would save memory, mark the object as - * incompressible so that we will not try to compress - * it again. + * in a way that would save memory. * - * We need to make sure that all secondary algorithms have - * failed, so we test if the number of recompressions matches - * the number of active secondary algorithms. + * Mark the object incompressible if the max-priority + * algorithm couldn't re-compress it. */ - if (num_recomps == zram->num_active_comps - 1) + if (prio < zram->num_active_comps) + return 0; + if (zram_test_flag(zram, index, ZRAM_PP_SLOT)) zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE); return 0; } - /* Successful recompression but above threshold */ - if (threshold && comp_len_new >= threshold) + /* Slot has been modified concurrently */ + if (!zram_test_flag(zram, index, ZRAM_PP_SLOT)) { + zcomp_stream_put(zram->comps[prio], zstrm); return 0; + } - /* - * If we cannot alloc memory for recompressed object then we bail out - * and simply keep the old (existing) object in zsmalloc. - */ + /* zsmalloc handle allocation can schedule, unlock slot's bucket */ + zram_slot_write_unlock(zram, index); handle_new = zs_malloc(zram->mem_pool, comp_len_new, GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); + zram_slot_write_lock(zram, index); + + /* + * If we couldn't allocate memory for recompressed object then bail + * out and simply keep the old (existing) object in mempool. + */ if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zram->comps[prio], zstrm); return PTR_ERR((void *)handle_new); } + /* Slot has been modified concurrently */ + if (!zram_test_flag(zram, index, ZRAM_PP_SLOT)) { + zcomp_stream_put(zram->comps[prio], zstrm); + zs_free(zram->mem_pool, handle_new); + return 0; + } + dst = zs_map_object(zram->mem_pool, handle_new, ZS_MM_WO); memcpy(dst, zstrm->buffer, comp_len_new); zcomp_stream_put(zram->comps[prio], zstrm);