From patchwork Mon Jan 27 07:29:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13951021 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38B38C0218C for ; Mon, 27 Jan 2025 07:29:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BF7A1280120; Mon, 27 Jan 2025 02:29:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BA7842800DA; Mon, 27 Jan 2025 02:29:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A6F91280120; Mon, 27 Jan 2025 02:29:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E65432800DA for ; Mon, 27 Jan 2025 02:29:50 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4B24081FF9 for ; Mon, 27 Jan 2025 07:29:50 +0000 (UTC) X-FDA: 83052407340.03.E6072D4 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf24.hostedemail.com (Postfix) with ESMTP id 5069A180004 for ; Mon, 27 Jan 2025 07:29:47 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=PfnO+aPv; spf=pass (imf24.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.177 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737962987; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=q566LymSR3yLFFaT+LJFKvYojLsB2mtA0so6QzTzlYU=; b=whX5i62oQ7N8AhWeFc9KBAaUeH9m4FI8njhhcI5HE4rFqUAktegXWhS4JOlcuiKFOGCd5Z yDX60AAFjJgTYRqS08QoO2+sETGceZ/Kap4Qsd5EE0vj5naBgmkJUpyE/ZSRBGC3/jO3mT X10nflTtIrtOpSnGZYqnhkzknVld4uY= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=PfnO+aPv; spf=pass (imf24.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.177 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737962987; a=rsa-sha256; cv=none; b=HoE5VvM/ZpNW6NQFvRBcmuCtqSHBSjp5IiQhd16cks6B/kpk0lKgIU9U+yYSJ18hTnrrib 9HhX7FzMEh30b5L51Aibu0JQWfEIMrLFw9ZusRpiZiVfichpdpeuoe1Z6gvnr+ZNzQIW8a iX5q260ihMiyRUSg0YiWkocpkGQ7yuE= Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-215770613dbso49459295ad.2 for ; Sun, 26 Jan 2025 23:29:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1737962986; x=1738567786; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=q566LymSR3yLFFaT+LJFKvYojLsB2mtA0so6QzTzlYU=; b=PfnO+aPvX3vb0jEH8TXjkLjVK7EMVKv8CSSzWmn3Tqzz/kD4Q9UWmRpwEFj7iDk7wE fp8YCPOK4D/N14bjaSnSGmZwX4C0fFlBKiCrbQNwHWmc5xF07PmjXix+nXF9q0S+3KW2 Hu/+WMh0a7N8iY1NkS6ABFCkjkviElfh+uh6k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737962986; x=1738567786; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=q566LymSR3yLFFaT+LJFKvYojLsB2mtA0so6QzTzlYU=; b=IV0nNNuRs/PAWM0GI8/mEKtZaOyfYxIpCFmNEfL8cc1S3rauAawpa0K58IWKSie+4q uR0ZlTSDU7MSdRMHCjwMoaSqNjQoQx+Vum3l/WjLWa7V4JlMjjqZTwBYB7m/afH1yinR s1QfMtB8r99XzDMi1ol1HMpib0n4WXws+B7ZUA+/pgpO6m5O/rZKrhPIa5CnJIfof+2D AuIhzdw5MOTdCvKVShJnYdiQAPtR9ILxRCN/HDLqspoKrzN/TRMD87kx4DoamDY/ql1e tDh7fjVhKGP6hv9hpIzXdL18T8tjEh2chhpnW9IOHNYFR/EurPn66SPzxINPvXEmOcGQ nWOg== X-Gm-Message-State: AOJu0Yze3CWAvfLxmoPMydUi8enmlHm3XV4qH/16immDGHtw2qDP/SqT Zymh0XfymQwr7SNvdO6m1cDb+ORo+P3b96/QQQywb++D6lRKFXRXjgURIHMZNg== X-Gm-Gg: ASbGncuuBdlT95jlOQAwiY2RC8duifkKP79uNonvW/QSd/buM04bS3AXB0mZfY6tNDm XetV99jm6EWCj2lgctaE7hAEIc0aAT/dAdIfGKxl3GGrtOxt2fpDSgUDz1GxMttUBwsR+rw0SPf fcBuepWQ3SGVVUapGvxwiUan3S9BBMH4fVkJc17575cGxuIr/uH+rx+fUItUUmaMrC5ui/U30IB S4SpsZhCSVncffUJ4GjMRHrLE95o7ErifUsP+YvgHSOmbKAgC4HZMnCFlVavTaKpvv2Xoy23iOJ Z5lonSA= X-Google-Smtp-Source: AGHT+IHWGcaKLW0ar/9axezARL9DLtAwfcGy5WchQ6brciU9TbvtbSpVRLYsS13AacDLWO3sj1eyFw== X-Received: by 2002:a05:6a00:8013:b0:725:8c0f:6fa3 with SMTP id d2e1a72fcca58-72dafbaae38mr52843068b3a.22.1737962986053; Sun, 26 Jan 2025 23:29:46 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:566d:6152:c049:8d3a]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72f8a69f41esm6319379b3a.31.2025.01.26.23.29.44 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 26 Jan 2025 23:29:45 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 1/9] zram: switch to non-atomic entry locking Date: Mon, 27 Jan 2025 16:29:12 +0900 Message-ID: <20250127072932.1289973-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250127072932.1289973-1-senozhatsky@chromium.org> References: <20250127072932.1289973-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 5069A180004 X-Stat-Signature: raabz6yinm4pfrxz9wf6w5mhnjduckm5 X-Rspam-User: X-HE-Tag: 1737962987-904273 X-HE-Meta: U2FsdGVkX18uJnOzoFPqPv0kDvuFR7lL3Nym1Ie+z5UniC9k20eNXoaeYuIkzvBl2UcR0zV4+49zZOHXXyiR+830p+3t9Fnig4QcGnfm5y8HOkFlcdjaqgptzHLFVmpQyutDeg04D1Sx6eos19tDLQ4fUF2vxGJvqVHoz+7zA+VBCdOc0Tu7xZ3EdGTykYZMJRi3iU2H197YU9eBNz3I/bNmris6Dq/2kxuUUFhzYJrbzrriPel9Q0kky7KAzpPLDyONhxQQWTdvoeVY3gVu8fNlW7+IU6qIci7AaPnbpGFR7k4zZYS2RXn9cXqGsS01gJOcKiVZgaAW8j/9Kvg8WAGY/dkJH9InawaiYBXrSWiSspi7UNXQyWeipM1EMTOI2E1DzAQwU1A57nWk0gLLrHqzVBzn0Z3rl4VBkJ8CxG7GaBehH0G2LzTQmrKiXkSIofngecWZqjNgHQeEXSEwYuIpQuYGJ1r6rzXlzOKSNwggC8BTV/Oz5VDK7PNdIItcFR7dapiCVxx1ezzmGCCNt1FzDZ6JPwEi5U+raInV9La2NorUUrjbc0gfG7nAdoXJNRW6oOsNfkRzhYmwzx7scCmugoapDIEEXLoYu1/rMcIGY/pN45iixPvdYS7lYIG89qdh0gphB8ZeQcdiMIxAnY5ECiPtDNDOxWgiHamSU/WHbK6NjfY2Orfx1wklcUURUjuDGh/jXR2bgA874StWxcL6xU0MjINTrmaPytNMjYXD7Gvi8Q6mdBxC0PBzzp6WujNMf0mUl76FnzMVmmTiE1vLTb5SU43F7o6r8jbw47K6bnrZ93XTRdt747UkW0I1onv0Ly1IJ1gTifaR+BrD5qo2G985FvZ7Y27S3EVlpmqktVMHJRUBMcgZkYV0lZaPZUT1RgmV/eooyUOki2/wN1qMrH81oALIdn9gW667HeqnLI8NAG5nECi6stmXpoyll8w3VENq/lT0YOX6RLW n/Zwp9oW sIzsOLr6653R02CXP153o88ReJShgiseWKJJTPXHaHheYKu9PJkpdrnr3R3p1BhPYa2fNVj+6QKy8rZbsDazZhgVw/1MrRwSqXC/HUwjXPtWqHxhzaTJjhLsvtxwrqVCgd+pqy7f22PyVCqGrpl2WlZp/c2bfnFreMDkxiOo6sv66Y+REISkN0pldF26nWt5YNEnWUziNtjtIXGQQIqbenO1dpbwu4vEZjhQcxq5Q79qPRtXFzBGTAvhIsOwT3GAHvmqgAxbuk4wkcsA3tNIWWUfS1pxBdC5up83eXxrUHcGINVfMcT3S5HCsrA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Concurrent modifications of meta table entries is now handled by per-entry spin-lock. This has a number of shortcomings. First, this imposes atomic requirements on compression backends. zram can call both zcomp_compress() and zcomp_decompress() under entry spin-lock, which implies that we can use only compression algorithms that don't schedule/sleep/wait during compression and decompression. This, for instance, makes it impossible to use some of the ASYNC compression algorithms (H/W compression, etc.) implementations. Second, this can potentially trigger watchdogs. For example, entry re-compression with secondary algorithms is performed under entry spin-lock. Given that we chain secondary compression algorithms and that some of them can be configured for best compression ratio (and worst compression speed) zram can stay under spin-lock for quite some time. Do not use per-entry spin-locks and instead convert it to an atomic_t variable which open codes reader-writer type of lock. This permits preemption from slot_lock section. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 155 +++++++++++++++++++++------------- drivers/block/zram/zram_drv.h | 7 +- 2 files changed, 98 insertions(+), 64 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 9f5020b077c5..14859bd2611f 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -58,19 +58,57 @@ static void zram_free_page(struct zram *zram, size_t index); static int zram_read_from_zspool(struct zram *zram, struct page *page, u32 index); -static int zram_slot_trylock(struct zram *zram, u32 index) +static int zram_slot_write_trylock(struct zram *zram, u32 index) { - return spin_trylock(&zram->table[index].lock); + int old; + + old = atomic_cmpxchg(&zram->table[index].lock, ZRAM_ENTRY_UNLOCKED, + ZRAM_ENTRY_WRLOCKED); + return old == ZRAM_ENTRY_UNLOCKED; } -static void zram_slot_lock(struct zram *zram, u32 index) +static void zram_slot_write_lock(struct zram *zram, u32 index) { - spin_lock(&zram->table[index].lock); + atomic_t *lock = &zram->table[index].lock; + int old; + + while (1) { + old = atomic_cmpxchg(lock, ZRAM_ENTRY_UNLOCKED, + ZRAM_ENTRY_WRLOCKED); + if (old == ZRAM_ENTRY_UNLOCKED) + return; + + cond_resched(); + } } -static void zram_slot_unlock(struct zram *zram, u32 index) +static void zram_slot_write_unlock(struct zram *zram, u32 index) { - spin_unlock(&zram->table[index].lock); + atomic_set(&zram->table[index].lock, ZRAM_ENTRY_UNLOCKED); +} + +static void zram_slot_read_lock(struct zram *zram, u32 index) +{ + atomic_t *lock = &zram->table[index].lock; + int old; + + while (1) { + old = atomic_read(lock); + if (old == ZRAM_ENTRY_WRLOCKED) { + cond_resched(); + continue; + } + + if (atomic_cmpxchg(lock, old, old + 1) == old) + return; + + cond_resched(); + } +} + +static void zram_slot_read_unlock(struct zram *zram, u32 index) +{ + atomic_dec(&zram->table[index].lock); } static inline bool init_done(struct zram *zram) @@ -93,7 +131,6 @@ static void zram_set_handle(struct zram *zram, u32 index, unsigned long handle) zram->table[index].handle = handle; } -/* flag operations require table entry bit_spin_lock() being held */ static bool zram_test_flag(struct zram *zram, u32 index, enum zram_pageflags flag) { @@ -229,9 +266,9 @@ static void release_pp_slot(struct zram *zram, struct zram_pp_slot *pps) { list_del_init(&pps->entry); - zram_slot_lock(zram, pps->index); + zram_slot_write_lock(zram, pps->index); zram_clear_flag(zram, pps->index, ZRAM_PP_SLOT); - zram_slot_unlock(zram, pps->index); + zram_slot_write_unlock(zram, pps->index); kfree(pps); } @@ -394,11 +431,11 @@ static void mark_idle(struct zram *zram, ktime_t cutoff) * * And ZRAM_WB slots simply cannot be ZRAM_IDLE. */ - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); if (!zram_allocated(zram, index) || zram_test_flag(zram, index, ZRAM_WB) || zram_test_flag(zram, index, ZRAM_SAME)) { - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); continue; } @@ -410,7 +447,7 @@ static void mark_idle(struct zram *zram, ktime_t cutoff) zram_set_flag(zram, index, ZRAM_IDLE); else zram_clear_flag(zram, index, ZRAM_IDLE); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); } } @@ -709,7 +746,7 @@ static int scan_slots_for_writeback(struct zram *zram, u32 mode, INIT_LIST_HEAD(&pps->entry); - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); if (!zram_allocated(zram, index)) goto next; @@ -731,7 +768,7 @@ static int scan_slots_for_writeback(struct zram *zram, u32 mode, place_pp_slot(zram, ctl, pps); pps = NULL; next: - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); } kfree(pps); @@ -822,7 +859,7 @@ static ssize_t writeback_store(struct device *dev, } index = pps->index; - zram_slot_lock(zram, index); + zram_slot_read_lock(zram, index); /* * scan_slots() sets ZRAM_PP_SLOT and relases slot lock, so * slots can change in the meantime. If slots are accessed or @@ -833,7 +870,7 @@ static ssize_t writeback_store(struct device *dev, goto next; if (zram_read_from_zspool(zram, page, index)) goto next; - zram_slot_unlock(zram, index); + zram_slot_read_unlock(zram, index); bio_init(&bio, zram->bdev, &bio_vec, 1, REQ_OP_WRITE | REQ_SYNC); @@ -860,7 +897,7 @@ static ssize_t writeback_store(struct device *dev, } atomic64_inc(&zram->stats.bd_writes); - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); /* * Same as above, we release slot lock during writeback so * slot can change under us: slot_free() or slot_free() and @@ -882,7 +919,7 @@ static ssize_t writeback_store(struct device *dev, zram->bd_wb_limit -= 1UL << (PAGE_SHIFT - 12); spin_unlock(&zram->wb_limit_lock); next: - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); release_pp_slot(zram, pps); cond_resched(); @@ -1001,7 +1038,7 @@ static ssize_t read_block_state(struct file *file, char __user *buf, for (index = *ppos; index < nr_pages; index++) { int copied; - zram_slot_lock(zram, index); + zram_slot_read_lock(zram, index); if (!zram_allocated(zram, index)) goto next; @@ -1019,13 +1056,13 @@ static ssize_t read_block_state(struct file *file, char __user *buf, ZRAM_INCOMPRESSIBLE) ? 'n' : '.'); if (count <= copied) { - zram_slot_unlock(zram, index); + zram_slot_read_unlock(zram, index); break; } written += copied; count -= copied; next: - zram_slot_unlock(zram, index); + zram_slot_read_unlock(zram, index); *ppos += 1; } @@ -1455,33 +1492,31 @@ static void zram_meta_free(struct zram *zram, u64 disksize) static bool zram_meta_alloc(struct zram *zram, u64 disksize) { - size_t num_pages, index; + size_t num_ents, index; - num_pages = disksize >> PAGE_SHIFT; - zram->table = vzalloc(array_size(num_pages, sizeof(*zram->table))); + num_ents = disksize >> PAGE_SHIFT; + zram->table = vzalloc(array_size(num_ents, sizeof(*zram->table))); if (!zram->table) - return false; + goto error; zram->mem_pool = zs_create_pool(zram->disk->disk_name); - if (!zram->mem_pool) { - vfree(zram->table); - zram->table = NULL; - return false; - } + if (!zram->mem_pool) + goto error; if (!huge_class_size) huge_class_size = zs_huge_class_size(zram->mem_pool); - for (index = 0; index < num_pages; index++) - spin_lock_init(&zram->table[index].lock); + for (index = 0; index < num_ents; index++) + atomic_set(&zram->table[index].lock, ZRAM_ENTRY_UNLOCKED); + return true; + +error: + vfree(zram->table); + zram->table = NULL; + return false; } -/* - * To protect concurrent access to the same index entry, - * caller should hold this table index entry's bit_spinlock to - * indicate this index entry is accessing. - */ static void zram_free_page(struct zram *zram, size_t index) { unsigned long handle; @@ -1602,17 +1637,17 @@ static int zram_read_page(struct zram *zram, struct page *page, u32 index, { int ret; - zram_slot_lock(zram, index); + zram_slot_read_lock(zram, index); if (!zram_test_flag(zram, index, ZRAM_WB)) { /* Slot should be locked through out the function call */ ret = zram_read_from_zspool(zram, page, index); - zram_slot_unlock(zram, index); + zram_slot_read_unlock(zram, index); } else { /* * The slot should be unlocked before reading from the backing * device. */ - zram_slot_unlock(zram, index); + zram_slot_read_unlock(zram, index); ret = read_from_bdev(zram, page, zram_get_handle(zram, index), parent); @@ -1655,10 +1690,10 @@ static int zram_bvec_read(struct zram *zram, struct bio_vec *bvec, static int write_same_filled_page(struct zram *zram, unsigned long fill, u32 index) { - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); zram_set_flag(zram, index, ZRAM_SAME); zram_set_handle(zram, index, fill); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); atomic64_inc(&zram->stats.same_pages); atomic64_inc(&zram->stats.pages_stored); @@ -1693,11 +1728,11 @@ static int write_incompressible_page(struct zram *zram, struct page *page, kunmap_local(src); zs_unmap_object(zram->mem_pool, handle); - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); zram_set_flag(zram, index, ZRAM_HUGE); zram_set_handle(zram, index, handle); zram_set_obj_size(zram, index, PAGE_SIZE); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); atomic64_add(PAGE_SIZE, &zram->stats.compr_data_size); atomic64_inc(&zram->stats.huge_pages); @@ -1718,9 +1753,9 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) bool same_filled; /* First, free memory allocated to this slot (if any) */ - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); zram_free_page(zram, index); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); mem = kmap_local_page(page); same_filled = page_same_filled(mem, &element); @@ -1790,10 +1825,10 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); zs_unmap_object(zram->mem_pool, handle); - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); zram_set_handle(zram, index, handle); zram_set_obj_size(zram, index, comp_len); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); /* Update stats */ atomic64_inc(&zram->stats.pages_stored); @@ -1850,7 +1885,7 @@ static int scan_slots_for_recompress(struct zram *zram, u32 mode, INIT_LIST_HEAD(&pps->entry); - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); if (!zram_allocated(zram, index)) goto next; @@ -1871,7 +1906,7 @@ static int scan_slots_for_recompress(struct zram *zram, u32 mode, place_pp_slot(zram, ctl, pps); pps = NULL; next: - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); } kfree(pps); @@ -2162,7 +2197,7 @@ static ssize_t recompress_store(struct device *dev, if (!num_recomp_pages) break; - zram_slot_lock(zram, pps->index); + zram_slot_write_lock(zram, pps->index); if (!zram_test_flag(zram, pps->index, ZRAM_PP_SLOT)) goto next; @@ -2170,7 +2205,7 @@ static ssize_t recompress_store(struct device *dev, &num_recomp_pages, threshold, prio, prio_max); next: - zram_slot_unlock(zram, pps->index); + zram_slot_write_unlock(zram, pps->index); release_pp_slot(zram, pps); if (err) { @@ -2217,9 +2252,9 @@ static void zram_bio_discard(struct zram *zram, struct bio *bio) } while (n >= PAGE_SIZE) { - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); zram_free_page(zram, index); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); atomic64_inc(&zram->stats.notify_free); index++; n -= PAGE_SIZE; @@ -2248,9 +2283,9 @@ static void zram_bio_read(struct zram *zram, struct bio *bio) } flush_dcache_page(bv.bv_page); - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); zram_accessed(zram, index); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); bio_advance_iter_single(bio, &iter, bv.bv_len); } while (iter.bi_size); @@ -2278,9 +2313,9 @@ static void zram_bio_write(struct zram *zram, struct bio *bio) break; } - zram_slot_lock(zram, index); + zram_slot_write_lock(zram, index); zram_accessed(zram, index); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); bio_advance_iter_single(bio, &iter, bv.bv_len); } while (iter.bi_size); @@ -2321,13 +2356,13 @@ static void zram_slot_free_notify(struct block_device *bdev, zram = bdev->bd_disk->private_data; atomic64_inc(&zram->stats.notify_free); - if (!zram_slot_trylock(zram, index)) { + if (!zram_slot_write_trylock(zram, index)) { atomic64_inc(&zram->stats.miss_free); return; } zram_free_page(zram, index); - zram_slot_unlock(zram, index); + zram_slot_write_unlock(zram, index); } static void zram_comp_params_reset(struct zram *zram) diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index db78d7c01b9a..3436ddf8ab23 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -15,7 +15,6 @@ #ifndef _ZRAM_DRV_H_ #define _ZRAM_DRV_H_ -#include #include #include @@ -28,7 +27,6 @@ #define ZRAM_SECTOR_PER_LOGICAL_BLOCK \ (1 << (ZRAM_LOGICAL_BLOCK_SHIFT - SECTOR_SHIFT)) - /* * ZRAM is mainly used for memory efficiency so we want to keep memory * footprint small and thus squeeze size and zram pageflags into a flags @@ -58,13 +56,14 @@ enum zram_pageflags { __NR_ZRAM_PAGEFLAGS, }; -/*-- Data structures */ +#define ZRAM_ENTRY_UNLOCKED 0 +#define ZRAM_ENTRY_WRLOCKED (-1) /* Allocated for each disk page */ struct zram_table_entry { unsigned long handle; unsigned int flags; - spinlock_t lock; + atomic_t lock; #ifdef CONFIG_ZRAM_TRACK_ENTRY_ACTIME ktime_t ac_time; #endif From patchwork Mon Jan 27 07:29:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13951022 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA845C02190 for ; Mon, 27 Jan 2025 07:29:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 36C9F280121; Mon, 27 Jan 2025 02:29:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 31D4A2800DA; Mon, 27 Jan 2025 02:29:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E400280121; Mon, 27 Jan 2025 02:29:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id F085D2800DA for ; Mon, 27 Jan 2025 02:29:54 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 7CE5D1C9A39 for ; Mon, 27 Jan 2025 07:29:54 +0000 (UTC) X-FDA: 83052407508.04.BB1C845 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf09.hostedemail.com (Postfix) with ESMTP id 9B181140002 for ; Mon, 27 Jan 2025 07:29:52 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=fFfOzGyo; spf=pass (imf09.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.52 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737962992; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=J0265o4fXtEfPILxc30ZTaArQv74vpUz9xkcp3pdNNI=; b=xn3FMyGqp3TQkK3ZoP2uYXpnd1S5mmRQf1rCANeqk59nw1gcmlAsQcG8QHeLj/0K3MUZqL N8DtZhkQ164O535JMP7XP0YpgKByl9M5RwxRkweUafZEXiPyKlQu2lF+tuizOUu8DrIHGd 6p4uz7P4Omu04Lc6+eCFZYHRyTieoEM= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=fFfOzGyo; spf=pass (imf09.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.52 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737962992; a=rsa-sha256; cv=none; b=YPkzNb7ZnssXe4eC7Y1Wpme+svg18JNjk9X6G66sfqt2vRgnagttcnhOPTvvGY2QmzqWPT ESDGYtV+owCeMZD9oD7vuLVWIBo0HsbsofdVD4kvookvlmsrqoAKCbrYpwFs6wzY04TePa RLqrgBW1MBZqmAb9uJA0Sv8jPsejjmU= Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2ef714374c0so6198015a91.0 for ; Sun, 26 Jan 2025 23:29:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1737962991; x=1738567791; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=J0265o4fXtEfPILxc30ZTaArQv74vpUz9xkcp3pdNNI=; b=fFfOzGyoP+5EOafEeHh1N/viEQCUb+xcw7zwwGldIlvaGsrevhnvQMcCtl87bpXnXc 4O2OfML025l3QHnPX+e6OeinY2nH47zPt907IGIzLF0KgHVukGLO0in+iV2lUMqhghNj is82900URdBTtRTDwmC6UBj2nn1FPkPW3pKHc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737962991; x=1738567791; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=J0265o4fXtEfPILxc30ZTaArQv74vpUz9xkcp3pdNNI=; b=C6WwEhtGUUYGiCV1yP7Xi4Xl6/uBWfRpDUmatr8Dgc8xdI4Ofsf+RR1KBf3ao36YXx wE01qNADNlcUJL+Pp9uCBc7CEToFFnNViBosxE+h4CT6SvplJHXTSZ3C9+QY0YMkwI3T fBxZ2xDgv6YQZsQZCMyfJMToZWOvf9owoX1sIan5GMcOQySIF+PqFfHonCZsJ/BHOTDp 2U+39DdrK4+rAHsdj5Ci/J/diOM0JOQUDsVDtVt+oOBR+Cb4HQAOwtRSQXmUmWoOzV4i CWRAUElsxF9FqqTZlOxmPvsZUf4UJbZJg5k04C8+0tfCY8i/e5pX03gYOzPO7B2wRyJf GEzw== X-Gm-Message-State: AOJu0YwhvyeYKFJvbYGFjGcD2ZiBeLITwo6judxN3B3DJQDtNVysifUd MH1qEhZBPJfRPw1snSP2275h7lfPL9Lm/REOcanjWW8PVs+mtiEAH0tAwuWXIw== X-Gm-Gg: ASbGnctapXVG+ZG5c8GWUoshycuuCL852TpIyQ18IzuivUbs7SIdjhyDUlQHRLRwDjY 2wF2o50CtR56EEXM5xW4bjEdmL3f03ZCiXhJ3/Y37sOBowrUzKPG+/OAoJex4rlIO9D95Z1n0Ts d/XJtaVBWLtQD6aVW6h48IWqck/gykQ+ygaTHx5yWwTz1J2dHu6ajB/cKzCdm0pbJaGLvjfXKiI 02egOJLkyFNLUc/TOABsOxL36eL0WHl2z1E7H014FkTO01yCjsVSHW+40ts8UhAv+P5gB5pLkHo GHjOjJU= X-Google-Smtp-Source: AGHT+IFSyrUoBu0dmirDsCNwBpWN0PP0zKexAyYjhwQ6BiECY+L1XRsV9WJ0VUAZ72LIigkZJ+PQTw== X-Received: by 2002:a17:90b:5488:b0:2ee:7e53:bfae with SMTP id 98e67ed59e1d1-2f7f177c6b3mr25160506a91.10.1737962991164; Sun, 26 Jan 2025 23:29:51 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:566d:6152:c049:8d3a]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2f7ffaf8f03sm6352175a91.39.2025.01.26.23.29.49 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 26 Jan 2025 23:29:50 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 2/9] zram: do not use per-CPU compression streams Date: Mon, 27 Jan 2025 16:29:13 +0900 Message-ID: <20250127072932.1289973-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250127072932.1289973-1-senozhatsky@chromium.org> References: <20250127072932.1289973-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 9B181140002 X-Stat-Signature: 49kpb8hm6qrnwhgd37xehy7odks3qffg X-Rspam-User: X-HE-Tag: 1737962992-362032 X-HE-Meta: U2FsdGVkX18Wb50R6+v/FS/o6mrBR+afe4DIA9RTHCKNpVP8MWNNUwmvPTJo7lQavpXaO+yh7Pprd6QbhfcMNX6DLnXAC00iQog6DN1Khgz8EkL5dCLGQkTtZotOWrqwU9Amk0AysEjkovNCofdKGsXqYZb/q2f9sTa30J7svPuWMkakEjv83T81H7562XyftuyAEP/AYbC112t05cF4CjKRiuojy1VajiLSlbPx3NUpnH0lMAzbp1AAo7qMdaPlKMrzsHtBelQA5i5/OUw8RMihpOZc8lNItqOJq4aWovfeg26kaM2PvBtFp4SdH2/i93PZPNoaFaV5sNftgCszqkZErPEm/etwQflIOHIgtmNI6uzoY0NdSU1axkerfO0PI5bufhYpA+qvK4ugNKafnV13nOYWCjQ6PKtzPbiuF6cwp36DSrjhPfwXultjJ+91SaOeClWnsRM05K6AGUWCwc5KAXhh6C5LX2mtMI5R2FiARiEabzt1t/dEIkO9OiPJE07L6emAa1cNF/dEC7b9Kz45DPGZd+s1Be1bbXYv4sDj16QRndYEt6lDnYsfb0Yrrx6V24z0IUxPeNuJZ6sDsuV05QRLdNJ76J6nmhrXESTHHv6Jd9N+i/0pCQpz6gUPYF68RtBEDtzXqKf0A0l1rzfPzteN6e97OxA4WaYLQu4ddEVqzZEgQWpx2v3cAZzHSqsSU82QCmVdH/4hxt0UVlRYVCsxTB+86SxtpWwKlbvI9WxCWDYYs0yZ/YPDAFo6zUuI+ISfbxMzUB/03LCGxeGD2Jw5bL1tAH4rCmkxdVV57gyqQ6U+Qvq9KT4gKfjWfV34CMJNMa8rruOl3D88dt2lYA4oMF7K/6GXlz2twdl3Bg8DM2Oz7B1kXgx0GCM3mJDBjVeSwwsbYrOp/nfmCON2HVCO5J5Wn/nFhRVeK3McLICGBBqpejzynhFIgfiL1AzkOqElo/sS7RYX+NN ktd4lDu/ SnEE+w7qA1ETVBIdA90URlo0QhROPW1PeAkKdVr6GYNK4BaNlnp9jo+vBfzZbaArVGMELbcmKysFP7Yy0o8ptr2IEJ550oUOZN7haTvxpYAPG9+9UzgmtUYSO9wD6/txaVoFXYoxIGMgPT6hiR1uoqFaNdXn4OpM+8iSmjLxE1Cs21RydzPcxx9VDMI31nsvZUbSSyZqzDZklQfElbrBAJWftjaAn1amwnk1xi3aQq1N9MjNnLgdHFCXr1z15zMKWBtUDFsIZ1BX7115TNMuYj0fOM/39SsslQt+B+cYJT6Gvxx24TV52gW5j+w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Similarly to per-entry spin-lock per-CPU compression streams also have a number of shortcoming. First, per-CPU stream access has to be done from a non-preemptible (atomic) section, which imposes the same atomicity requirements on compression backends as entry spin-lock do and makes it impossible to use algorithms that can schedule/wait/sleep during compression and decompression. Second, per-CPU streams noticeably increase memory usage (actually more like wastage) of secondary compression streams. The problem is that secondary compression streams are allocated per-CPU, just like the primary streams are. Yet we never use more that one secondary stream at a time, because recompression is a single threaded action. Which means that remaining num_online_cpu() - 1 streams are allocated for nothing, and this is per-priority list (we can have several secondary compression algorithms). Depending on the algorithm this may lead to a significant memory wastage, in addition each stream also carries a workmem buffer (2 physical pages). Instead of per-CPU streams, maintain a list of idle compression streams and allocate new streams on-demand (something that we used to do many years ago). So that zram read() and write() become non-atomic and ease requirements on the compression algorithm implementation. This also means that we now should have only one secondary stream per-priority list. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 164 +++++++++++++++++++--------------- drivers/block/zram/zcomp.h | 17 ++-- drivers/block/zram/zram_drv.c | 29 +++--- include/linux/cpuhotplug.h | 1 - 4 files changed, 109 insertions(+), 102 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index bb514403e305..982c769d5831 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -6,7 +6,7 @@ #include #include #include -#include +#include #include #include @@ -43,31 +43,40 @@ static const struct zcomp_ops *backends[] = { NULL }; -static void zcomp_strm_free(struct zcomp *comp, struct zcomp_strm *zstrm) +static void zcomp_strm_free(struct zcomp *comp, struct zcomp_strm *strm) { - comp->ops->destroy_ctx(&zstrm->ctx); - vfree(zstrm->buffer); - zstrm->buffer = NULL; + comp->ops->destroy_ctx(&strm->ctx); + vfree(strm->buffer); + kfree(strm); } -static int zcomp_strm_init(struct zcomp *comp, struct zcomp_strm *zstrm) +static struct zcomp_strm *zcomp_strm_alloc(struct zcomp *comp) { + struct zcomp_strm *strm; int ret; - ret = comp->ops->create_ctx(comp->params, &zstrm->ctx); - if (ret) - return ret; + strm = kzalloc(sizeof(*strm), GFP_KERNEL); + if (!strm) + return NULL; + + INIT_LIST_HEAD(&strm->entry); + + ret = comp->ops->create_ctx(comp->params, &strm->ctx); + if (ret) { + kfree(strm); + return NULL; + } /* - * allocate 2 pages. 1 for compressed data, plus 1 extra for the - * case when compressed size is larger than the original one + * allocate 2 pages. 1 for compressed data, plus 1 extra in case if + * compressed data is larger than the original one. */ - zstrm->buffer = vzalloc(2 * PAGE_SIZE); - if (!zstrm->buffer) { - zcomp_strm_free(comp, zstrm); - return -ENOMEM; + strm->buffer = vzalloc(2 * PAGE_SIZE); + if (!strm->buffer) { + zcomp_strm_free(comp, strm); + return NULL; } - return 0; + return strm; } static const struct zcomp_ops *lookup_backend_ops(const char *comp) @@ -109,13 +118,59 @@ ssize_t zcomp_available_show(const char *comp, char *buf) struct zcomp_strm *zcomp_stream_get(struct zcomp *comp) { - local_lock(&comp->stream->lock); - return this_cpu_ptr(comp->stream); + struct zcomp_strm *strm; + + might_sleep(); + + while (1) { + spin_lock(&comp->strm_lock); + if (!list_empty(&comp->idle_strm)) { + strm = list_first_entry(&comp->idle_strm, + struct zcomp_strm, + entry); + list_del(&strm->entry); + spin_unlock(&comp->strm_lock); + return strm; + } + + /* cannot allocate new stream, wait for an idle one */ + if (comp->avail_strm >= num_online_cpus()) { + spin_unlock(&comp->strm_lock); + wait_event(comp->strm_wait, + !list_empty(&comp->idle_strm)); + continue; + } + + /* allocate new stream */ + comp->avail_strm++; + spin_unlock(&comp->strm_lock); + + strm = zcomp_strm_alloc(comp); + if (strm) + break; + + spin_lock(&comp->strm_lock); + comp->avail_strm--; + spin_unlock(&comp->strm_lock); + wait_event(comp->strm_wait, !list_empty(&comp->idle_strm)); + } + + return strm; } -void zcomp_stream_put(struct zcomp *comp) +void zcomp_stream_put(struct zcomp *comp, struct zcomp_strm *strm) { - local_unlock(&comp->stream->lock); + spin_lock(&comp->strm_lock); + if (comp->avail_strm <= num_online_cpus()) { + list_add(&strm->entry, &comp->idle_strm); + spin_unlock(&comp->strm_lock); + wake_up(&comp->strm_wait); + return; + } + + comp->avail_strm--; + spin_unlock(&comp->strm_lock); + zcomp_strm_free(comp, strm); } int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, @@ -148,61 +203,19 @@ int zcomp_decompress(struct zcomp *comp, struct zcomp_strm *zstrm, return comp->ops->decompress(comp->params, &zstrm->ctx, &req); } -int zcomp_cpu_up_prepare(unsigned int cpu, struct hlist_node *node) -{ - struct zcomp *comp = hlist_entry(node, struct zcomp, node); - struct zcomp_strm *zstrm; - int ret; - - zstrm = per_cpu_ptr(comp->stream, cpu); - local_lock_init(&zstrm->lock); - - ret = zcomp_strm_init(comp, zstrm); - if (ret) - pr_err("Can't allocate a compression stream\n"); - return ret; -} - -int zcomp_cpu_dead(unsigned int cpu, struct hlist_node *node) -{ - struct zcomp *comp = hlist_entry(node, struct zcomp, node); - struct zcomp_strm *zstrm; - - zstrm = per_cpu_ptr(comp->stream, cpu); - zcomp_strm_free(comp, zstrm); - return 0; -} - -static int zcomp_init(struct zcomp *comp, struct zcomp_params *params) -{ - int ret; - - comp->stream = alloc_percpu(struct zcomp_strm); - if (!comp->stream) - return -ENOMEM; - - comp->params = params; - ret = comp->ops->setup_params(comp->params); - if (ret) - goto cleanup; - - ret = cpuhp_state_add_instance(CPUHP_ZCOMP_PREPARE, &comp->node); - if (ret < 0) - goto cleanup; - - return 0; - -cleanup: - comp->ops->release_params(comp->params); - free_percpu(comp->stream); - return ret; -} - void zcomp_destroy(struct zcomp *comp) { - cpuhp_state_remove_instance(CPUHP_ZCOMP_PREPARE, &comp->node); + struct zcomp_strm *strm; + + while (!list_empty(&comp->idle_strm)) { + strm = list_first_entry(&comp->idle_strm, + struct zcomp_strm, + entry); + list_del(&strm->entry); + zcomp_strm_free(comp, strm); + } + comp->ops->release_params(comp->params); - free_percpu(comp->stream); kfree(comp); } @@ -229,7 +242,12 @@ struct zcomp *zcomp_create(const char *alg, struct zcomp_params *params) return ERR_PTR(-EINVAL); } - error = zcomp_init(comp, params); + INIT_LIST_HEAD(&comp->idle_strm); + init_waitqueue_head(&comp->strm_wait); + spin_lock_init(&comp->strm_lock); + + comp->params = params; + error = comp->ops->setup_params(comp->params); if (error) { kfree(comp); return ERR_PTR(error); diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h index ad5762813842..62330829db3f 100644 --- a/drivers/block/zram/zcomp.h +++ b/drivers/block/zram/zcomp.h @@ -3,10 +3,10 @@ #ifndef _ZCOMP_H_ #define _ZCOMP_H_ -#include - #define ZCOMP_PARAM_NO_LEVEL INT_MIN +#include + /* * Immutable driver (backend) parameters. The driver may attach private * data to it (e.g. driver representation of the dictionary, etc.). @@ -31,7 +31,7 @@ struct zcomp_ctx { }; struct zcomp_strm { - local_lock_t lock; + struct list_head entry; /* compression buffer */ void *buffer; struct zcomp_ctx ctx; @@ -60,16 +60,15 @@ struct zcomp_ops { const char *name; }; -/* dynamic per-device compression frontend */ struct zcomp { - struct zcomp_strm __percpu *stream; + struct list_head idle_strm; + spinlock_t strm_lock; + u32 avail_strm; + wait_queue_head_t strm_wait; const struct zcomp_ops *ops; struct zcomp_params *params; - struct hlist_node node; }; -int zcomp_cpu_up_prepare(unsigned int cpu, struct hlist_node *node); -int zcomp_cpu_dead(unsigned int cpu, struct hlist_node *node); ssize_t zcomp_available_show(const char *comp, char *buf); bool zcomp_available_algorithm(const char *comp); @@ -77,7 +76,7 @@ struct zcomp *zcomp_create(const char *alg, struct zcomp_params *params); void zcomp_destroy(struct zcomp *comp); struct zcomp_strm *zcomp_stream_get(struct zcomp *comp); -void zcomp_stream_put(struct zcomp *comp); +void zcomp_stream_put(struct zcomp *comp, struct zcomp_strm *strm); int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm, const void *src, unsigned int *dst_len); diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 14859bd2611f..486068dda395 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -31,7 +31,6 @@ #include #include #include -#include #include #include @@ -1610,7 +1609,7 @@ static int read_compressed_page(struct zram *zram, struct page *page, u32 index) ret = zcomp_decompress(zram->comps[prio], zstrm, src, size, dst); kunmap_local(dst); zs_unmap_object(zram->mem_pool, handle); - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zram->comps[prio], zstrm); return ret; } @@ -1771,14 +1770,14 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) kunmap_local(mem); if (unlikely(ret)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); pr_err("Compression failed! err=%d\n", ret); zs_free(zram->mem_pool, handle); return ret; } if (comp_len >= huge_class_size) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); return write_incompressible_page(zram, page, index); } @@ -1802,7 +1801,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); atomic64_inc(&zram->stats.writestall); handle = zs_malloc(zram->mem_pool, comp_len, GFP_NOIO | __GFP_HIGHMEM | @@ -1814,7 +1813,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) } if (!zram_can_store_page(zram)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); zs_free(zram->mem_pool, handle); return -ENOMEM; } @@ -1822,7 +1821,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); memcpy(dst, zstrm->buffer, comp_len); - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); zs_unmap_object(zram->mem_pool, handle); zram_slot_write_lock(zram, index); @@ -1981,7 +1980,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, kunmap_local(src); if (ret) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zram->comps[prio], zstrm); return ret; } @@ -1991,7 +1990,7 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, /* Continue until we make progress */ if (class_index_new >= class_index_old || (threshold && comp_len_new >= threshold)) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zram->comps[prio], zstrm); continue; } @@ -2049,13 +2048,13 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zram->comps[prio], zstrm); return PTR_ERR((void *)handle_new); } dst = zs_map_object(zram->mem_pool, handle_new, ZS_MM_WO); memcpy(dst, zstrm->buffer, comp_len_new); - zcomp_stream_put(zram->comps[prio]); + zcomp_stream_put(zram->comps[prio], zstrm); zs_unmap_object(zram->mem_pool, handle_new); @@ -2803,7 +2802,6 @@ static void destroy_devices(void) zram_debugfs_destroy(); idr_destroy(&zram_index_idr); unregister_blkdev(zram_major, "zram"); - cpuhp_remove_multi_state(CPUHP_ZCOMP_PREPARE); } static int __init zram_init(void) @@ -2813,15 +2811,9 @@ static int __init zram_init(void) BUILD_BUG_ON(__NR_ZRAM_PAGEFLAGS > sizeof(zram_te.flags) * 8); - ret = cpuhp_setup_state_multi(CPUHP_ZCOMP_PREPARE, "block/zram:prepare", - zcomp_cpu_up_prepare, zcomp_cpu_dead); - if (ret < 0) - return ret; - ret = class_register(&zram_control_class); if (ret) { pr_err("Unable to register zram-control class\n"); - cpuhp_remove_multi_state(CPUHP_ZCOMP_PREPARE); return ret; } @@ -2830,7 +2822,6 @@ static int __init zram_init(void) if (zram_major <= 0) { pr_err("Unable to get major number\n"); class_unregister(&zram_control_class); - cpuhp_remove_multi_state(CPUHP_ZCOMP_PREPARE); return -EBUSY; } diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 6cc5e484547c..092ace7db8ee 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -119,7 +119,6 @@ enum cpuhp_state { CPUHP_MM_ZS_PREPARE, CPUHP_MM_ZSWP_POOL_PREPARE, CPUHP_KVM_PPC_BOOK3S_PREPARE, - CPUHP_ZCOMP_PREPARE, CPUHP_TIMERS_PREPARE, CPUHP_TMIGR_PREPARE, CPUHP_MIPS_SOC_PREPARE, From patchwork Mon Jan 27 07:29:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13951023 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54029C0218C for ; Mon, 27 Jan 2025 07:30:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D5C35280123; Mon, 27 Jan 2025 02:29:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CB99F2800DA; Mon, 27 Jan 2025 02:29:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A5407280123; Mon, 27 Jan 2025 02:29:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 81E0D2800DA for ; Mon, 27 Jan 2025 02:29:59 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 3B82A4B236 for ; Mon, 27 Jan 2025 07:29:59 +0000 (UTC) X-FDA: 83052407718.23.1F5BD5D Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) by imf24.hostedemail.com (Postfix) with ESMTP id 5E3FA18000E for ; Mon, 27 Jan 2025 07:29:57 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Y++ALsrT; spf=pass (imf24.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.51 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737962997; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZMKoDSDfvoX+go24fej91nvl1Fy002IINq4VDAWYIx0=; b=RdlZdx+wCA5koYFrx2MUAgDNOFfoXLVG8sGmjRLZxzVTy8djDGDwRavz0QqEER+WyNMpJ0 1plTacOh51pmQTAoyUlHv5BcCsXYJ+7fJ8AW9p1oI91lD8XJkyKkhKRlqvengViTMLy35+ oRZoEsD4sOYpWVdQ3Dfsd2NXmOY2PtQ= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Y++ALsrT; spf=pass (imf24.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.51 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737962997; a=rsa-sha256; cv=none; b=iquRYJro2am9VXj456mDNjF+GNN02zJy6vpQyZkafmc7P6T1Ky3iUF3mMsXgcjGhTea+xh gGbzWRo//qnGDEziz4Cp30PyixuFeNdqsbCVvhiDeZdIeheNUFN2WOa7qSyi9/bHZIo4qo Rb1sOwAqzTlP3w8il7zprN/bbjySpg4= Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-2efe25558ddso5339926a91.2 for ; Sun, 26 Jan 2025 23:29:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1737962996; x=1738567796; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZMKoDSDfvoX+go24fej91nvl1Fy002IINq4VDAWYIx0=; b=Y++ALsrTtT5Vb491cP8eos85Q2DeZOpA+/jo3PvkznG42m5Q0rLmphJBFiIJTMfEPU QLj+tLLYPzliilAuosPK6pUz+lYmvKAhu9xKETVfnT/9jxQvJ2SCh6jrfPDrr6OJmBp4 fBCura3IRn6WJCjRv8ZMnPGEQxHlhFxy073cs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737962996; x=1738567796; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZMKoDSDfvoX+go24fej91nvl1Fy002IINq4VDAWYIx0=; b=A2ZEEDpvlCGCaVkZ2+mQ0IUn+vJDfnNPG76+jK/4MQ+gd/BHmmmmgR7YvUzklBNUCZ z2G4R1BQP9WcjAKiYxxXBDY9tWSrSsaPZxw3oVTJQsFiSICvFzD96Hh0CY3RYRrW66sT GyfX3nohvCY2ttojEpo9XbAFQANC718XJsDBEQMWiwVFCVHFmQeRMSwLqVjzE/zdHTBb 6gBlBy25hhXDya8Fd/Vn1vzSN+gouu42PqlAI0CQ+kUhn09pkU+BzMIGwoePaPEiG+Y3 4xCg/QU3IgPQ9ZiD5ulJzf/1tdNsp+JZYdjAPr6GMoArcVL8A3ZmM5EXsgDsXUMccB/C tKsQ== X-Gm-Message-State: AOJu0Yz7+ZJ92vayQL9WUXTWjf/GE/OtH0HoU2mRL4NYCfXb8KbkbORR JMxGI/iy50JLJyFyPhcYNrzANhZu92mtR9H+fV+KQhsBO76UDrXXSuYTpOYwZg== X-Gm-Gg: ASbGncspWWmS+Nkd01UP5eQbupsjp0ZwUSV+T9dk6M8YqGDXi02BB+9a6kqo9qwiIiW LvMTPnEi45UJ5RlnXk52p0QenkD6z4nprwdoBOtIRmiXs9+Ag2LbkyOiTu1Jm/9I2b9Gv38IMbY R3WFcBl8AfyJtx//mPtzAqiBLpXzjndO/GZxkYJe3qBagjkmjnAhE79ByPDNPh+VJyGLJcwJt/U HictvzMMJGcuZ8EYMsBvkGzSgBLXgnVgeOPrht80rJWASBYrCFaZh+S71R/CU0v8V4/bZaCrEYA PePO060= X-Google-Smtp-Source: AGHT+IEfwUI8auttHRk1qh7AN8+U6mvQ/O5WBUL7JmvN0YslOiw0cRVxh68/cSFEaaq3Cgj0hlneng== X-Received: by 2002:a17:90b:2803:b0:2ee:ad18:b30d with SMTP id 98e67ed59e1d1-2f782c4bd4emr8745204a91.6.1737962996210; Sun, 26 Jan 2025 23:29:56 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:566d:6152:c049:8d3a]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2f7ffb19938sm6431742a91.45.2025.01.26.23.29.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 26 Jan 2025 23:29:55 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 3/9] zram: remove crypto include Date: Mon, 27 Jan 2025 16:29:14 +0900 Message-ID: <20250127072932.1289973-4-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250127072932.1289973-1-senozhatsky@chromium.org> References: <20250127072932.1289973-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 5E3FA18000E X-Stat-Signature: eza7z7b4peqgxr9qh4ngqf64aentnwu7 X-Rspam-User: X-HE-Tag: 1737962997-149593 X-HE-Meta: U2FsdGVkX18FKb55M8/fvZPGd0VE4FCYeX0FdzhySTTs1xDloFuL8Nw6rM3Gw6WaGkuxQy9XvWYh/TaMSaJ7Qg7kJetprbxBGvN7LgGAtVDxlOI623xwyG/NPJCBp5MROAbxbS2fPXseH5oQRMTObzJWmsPk4uxtwInyQDYyMa5KWmXO3Cc6eWrcLUZ5fV6p9uhePJrKgp0FsZ+oF6mIDYK+uYet0BWoiGlY78s7TdR2HX60XxrPPjQzQBnxH1jT0ciaMQ8FMwMXOaHxI/BqIFRAtkcfJ6d9j1UdwdGy4YyK6C4gxEWkPOFCl1B5b1RgPMJQ53EYmfrvZR6UWY/Bau5fdJ8AXBmvqvvHGpa3P3JTy9UPdqBaSoV89y3Cqg/w5FbqkdIwM2LDOdy7HfAS+sKgPCy6MqgliGy4wSG3+2LjnO3/gA4Uc2YGA0/vHkQ9C/FKN99jfrwIRfpscw9rkkEWFn+V/D6TfxdEzJln3mS2vyTLCVcLDovknuCIqfF24u2WmXv7yMbFygxhVDiyrsM5Te+1sJiEv4rHOdS8Q/0gTF1wMnZpue1zicY0n226SLFrkNg5qLEMsmMXDIaBOU8ElWVGyVvChBAQTVBQhQCD8FtCms+zEbE5u8NBQxawjMxDSHw5LfenQkrMtjdbck70UTMQU6GP3usDqgSOmkfOjHRzZ5jKN3v2ofMgCXCCXISB0zaGdDubwX8IEWvbpxvTBNmMFEGFGns2TPUNYF+MIGjeb4UNcoBoSTTSxQEGItFBMz0ow5RbJNnVzLXH3rVL+mnG/uLuyQtK4LBBTyshf7WKA2OUrrw82/VLNY23R3lgDKb2gfpzxuMwu8EsMwj7nYgNVTn6zkwwERY/qvr1IZXidyWnUs5oYxtQ0bEWaCU4m6JC9dOC2NKQzQ8dmmEFYJ2hSMjTz17M+XRhwrVHG+gxwNOQu1ySgqOHZy8BIVuyC9gtnxe+pn5yGUY SOAR1bZw Vh5ckg7jfQlkQZt/+vphQdnvv5S2OezJOCKRm7eXMCm3v1mazBC97qpEbk6wIXtEh8eYe/4T8GvxX6Np4QZ3TFeea9rg7ec9ZFKWsWoy/tmxYNaiilcOlT/TdT2EfSiRlS6g9orUNAzozJC5XyGKqpBa1oWF3nBtByh/uOUOvzL3chIobJCO2u8NAeoZMwGR+h6loRPDNKOVodtGAeloj0m5RW5dsqgPxKqHtTcAVTP9Y+bYEo0kuRIVKojJmstNObpoOCoQjC1C3EYCF43V+HQQk7RviTCa+HGmqVaEiq9Svqv6AuwseWgFSHnG+pUcBUJYMbRSKS+l4rKf36E3gzgki+g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Remove a leftover crypto header include. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zcomp.c | 1 - drivers/block/zram/zram_drv.c | 4 +++- drivers/block/zram/zram_drv.h | 1 - 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c index 982c769d5831..efd5919808d9 100644 --- a/drivers/block/zram/zcomp.c +++ b/drivers/block/zram/zcomp.c @@ -7,7 +7,6 @@ #include #include #include -#include #include #include "zcomp.h" diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 486068dda395..c1ecaac0b39f 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -43,6 +43,8 @@ static DEFINE_MUTEX(zram_index_mutex); static int zram_major; static const char *default_compressor = CONFIG_ZRAM_DEF_COMP; +#define ZRAM_MAX_ALGO_NAME_SZ 64 + /* Module params (documentation at end) */ static unsigned int num_devices = 1; /* @@ -1148,7 +1150,7 @@ static int __comp_algorithm_store(struct zram *zram, u32 prio, const char *buf) size_t sz; sz = strlen(buf); - if (sz >= CRYPTO_MAX_ALG_NAME) + if (sz >= ZRAM_MAX_ALGO_NAME_SZ) return -E2BIG; compressor = kstrdup(buf, GFP_KERNEL); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 3436ddf8ab23..045032acb22f 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -16,7 +16,6 @@ #define _ZRAM_DRV_H_ #include -#include #include "zcomp.h" From patchwork Mon Jan 27 07:29:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13951024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2D86C0218F for ; Mon, 27 Jan 2025 07:30:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2A89E280124; Mon, 27 Jan 2025 02:30:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 25AA02800DA; Mon, 27 Jan 2025 02:30:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0D2C2280124; Mon, 27 Jan 2025 02:30:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D94BF2800DA for ; Mon, 27 Jan 2025 02:30:03 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 95FF3AE3F5 for ; Mon, 27 Jan 2025 07:30:03 +0000 (UTC) X-FDA: 83052407886.02.F92670B Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf23.hostedemail.com (Postfix) with ESMTP id BD65714000D for ; Mon, 27 Jan 2025 07:30:01 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=MSsPWciZ; spf=pass (imf23.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737963001; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lUvRyh0me1bXpwqfK+TRyFS4H2+UkCD7EBcpdcldqV8=; b=s5i0mndaAY5Bz+G/LE37gNmnWlnKkuYAOAAb24il0pXBPXTW7YTNVrO3ez6mZs83V2xzIR QeLQy8ElQMBxZnloho+AMXEFQE1X/3xYYTvXFfRped9PkFQdb97FD1gOqdX+wdcrid59yg Jl0csxSobNa2HYkbyPzPvRWB+CIB6MY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737963001; a=rsa-sha256; cv=none; b=y9/obj6iBeHF3e9RtPzvnVENY9px/4260WyTWcDCbTpI5xE6Q98AqS5Z22Hy7jBc1EtVQo 1CO+8vwSTgVyi/pTi1lvsDwsaB4wZ8bXAZ+s5jGyasuuNfHb1K/Zj1Y9ozscGDcMoHnKWG zfH1YWPX8LbZOW0iPv+DIkUhxXEJv1I= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=MSsPWciZ; spf=pass (imf23.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-21675fd60feso90565215ad.2 for ; Sun, 26 Jan 2025 23:30:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1737963000; x=1738567800; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lUvRyh0me1bXpwqfK+TRyFS4H2+UkCD7EBcpdcldqV8=; b=MSsPWciZbSqSu8ngY9/E0Gb8hKsVFEar/dKQ7MSn9GuN4yTqItsR+RDSDd0VP0/C2r cNCy+v70LPeNJe8TyxnD08U6rWu0XyoFqo2saBBfPMHVqeLBrTX4xYxp2BALUu8eb9/9 MAawD28xTsA/WcuqjT+ArGGhtNHYw8y8Mg+pg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737963000; x=1738567800; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lUvRyh0me1bXpwqfK+TRyFS4H2+UkCD7EBcpdcldqV8=; b=cV97Jl68Nsy5kK3tJq0xdtcA7W6YQyHkdCXYW+fNy5I2M4MvIGAJgYnpsnsw4BO5zl EI1MKDdmeK0RXTM/UW7ZWV92Q08k/zrixEd9u7M/4HBFQRb14Rz2ixWMltNeMOmQoeIh i1MNYItvSwPMQ/1tCI12J+Zejnit/agyLzSu48J6gyi5LOdlH0qGAOTPiuTSpONIXrhS RzRRcvHs4uJg0VvZ7xLUt6h/928VAyj01fbZ4qSXDfqq2qCZlFypYxjs0kFroP4KIbBs 0sfvOiYULgbwJal9SJnv2j3braj79DtZXam2LoHYx+bhoE38dvc4/ZvHcdfHV2rVKFNH WJVg== X-Gm-Message-State: AOJu0YwV1+Q/YwIjYHuNl/Y5MiC1OoxbdplFcsN/QE0PZtN6IMEXV8nX Ox6FLpDRBka39ICt3Kmea9yLxnOdPyG90A5G2e5Bg3GRmZ9lGORUX5Ezw1pGrw== X-Gm-Gg: ASbGncuwjXLrdycvPXyVTpNPgJAUzaGT+SLBkXJpszrWoPc4c2ZcZHfTMtDcVMTD3wO YTA0NK39bQw5ckEs84aCFG6uWpCq0pWNFiduwdk0/63hTj2X7pW6lLedDAVAT8xVBdmlBTo/OEN 3qIQtnrHGishCxUWFs3BDFTTeMLZU2xpfm9f/cEqkl5Ne+2VuD20BhRQiosHUN/docIwknqyBp/ 7aQcAjPr84jc06YzAHcIPAjGg/KD6nkbsfXHOmmFAu88jfjxdxeiOVoEnuugxVETgmqtfRNeWso 1EoAocA= X-Google-Smtp-Source: AGHT+IEMZNai1/9yM8pfA7fL3mLUFYeR0CMWjd6TA0xHnzwmsWa0/+qcJq0oMDnr/ZCTRC5XkIHMLQ== X-Received: by 2002:a17:902:ea03:b0:219:d28a:ca23 with SMTP id d9443c01a7336-21c355eea74mr607842355ad.36.1737963000524; Sun, 26 Jan 2025 23:30:00 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:566d:6152:c049:8d3a]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-21da424e779sm56851105ad.213.2025.01.26.23.29.58 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 26 Jan 2025 23:30:00 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 4/9] zram: remove max_comp_streams device attr Date: Mon, 27 Jan 2025 16:29:15 +0900 Message-ID: <20250127072932.1289973-5-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250127072932.1289973-1-senozhatsky@chromium.org> References: <20250127072932.1289973-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Stat-Signature: skg159jzwbbmtydduhmgdrnkpzqo3ggs X-Rspam-User: X-Rspamd-Queue-Id: BD65714000D X-Rspamd-Server: rspam03 X-HE-Tag: 1737963001-281933 X-HE-Meta: U2FsdGVkX19ChYXJY03F/FXJC3e/SpgJLUmR7mfnEO9w3sihF4oEliExoMH6bNCRxo3w3SIN2yb0Gk4K9t+/i7hMXU2nJ3MOvG9sEcLvBNWkXJoOy6XiiI810HvPIGfoht4MgwSGy+cYAer06E1i0tTheN53bzaZ+eQFUiw7qdV/UcVzhvYjnBsfgi7YVeI1kvPOEh54LfsGvhWEPnRzE1QHVvlXQVgZSDoXg4hgTGLHEpjoWqNRcSXslfAuNTUrS3lrBMTHQwSTaWHukftsvkewwZIOXU8iabPdUY2m8A4ifbQtgssVevVC+3E0XPWHjpS2HGzz4ar1k9yJWos74XareZIyvpWQmPPOeWKcl9E6XSO9sMhfnbJAjYu70mw0//+NuHZjxoJkKc9h63Vt96fRfyC1Kp7nYpTvJal6TXEFPTXBm66FVqKY50X+WMiJCv/HWWmIdQ0RvXqIVpm9QfsMrBzt/lGvCFV7ErxM3lEdIIi8/mCsgUcr5YfczmAJm6zHMDxZ5MrgsNeGMszw95x7GqNAJvuNcRyKT/lCFM3BSyNTJwy0TRrab8ybnhkmwWVdIQ4pfbtuJbAgtfEXS5lHh6u87LkFQjQSIyDkMBOSkN7ID4SlrXaLNOL2eNoU4i6ugf8NiSfgBH8ZPgrbVU9bQQ15S6da8qMUHJea22iIyp9wWl1r3KQv0Ye3lEN3DjEcAjfPys4vb+HdEyCOk+vcZ9uRSteZIjX8mE3/zPJXcFP7XsZd2nZ0nWEhhLzYu911n8P1fDg4DETD/RXX3nxFuVaX+8x4DiD+7wqQ5zNlY4mZa61e4T6Nn1xUpN9mrtcyQ8Dr/DQbD9BqcPH1udaeY0cAhTGPZedIpmkyCVVVpp/dmw3ZuMWtS56s2lmCnQlimv20uqq0oIHi4cZuP2JIdOLnD9Lr8iDrcy506LYZeFGBgC+MD+0I/WeCpb2VDsbTpdxL43rQR+kmXu6 PQ+5E/3V A6+Re9jhc6mj9QZS+uLlrsGFSwcsWylzkDgj1hHVmt4cfU5HgQaY/OAUAqTw82OI5zKFGhuYugwRtujofRF2NsSbvN1BdiUUTH6FTwl0SEMSaL3qhv7yO6uEHCSAm3GQqR0Y4RRGKpdHvXAQHoKMRGWJtAtmarZ0igix2+U8vWK4ogJsh0Yynh7Rjw+a6GeHudyX8wOTRDoOxPy0Z69NbCR/nJG/VQrJJ3aiJ6Ts7mPOWpQYjFKBC5jYyBnn3wU6u4h6wz/VjdKiPxxJY9iEev5ZElOmbw+agAQ35PIgLCV9mu05y174Dre8RQPzqQZ0Vw9Ylza6Qc8fBFuwq57q3FSTsyWye7Ysa4CuLUim0pD4Poh/x6EnFRtCt/mv9e/F6v0kn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: max_comp_streams device attribute has been defunct since May 2016 when zram switched to per-CPU compression streams, remove it. Signed-off-by: Sergey Senozhatsky --- Documentation/ABI/testing/sysfs-block-zram | 8 ----- Documentation/admin-guide/blockdev/zram.rst | 36 ++++++--------------- drivers/block/zram/zram_drv.c | 23 ------------- 3 files changed, 10 insertions(+), 57 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-block-zram b/Documentation/ABI/testing/sysfs-block-zram index 1ef69e0271f9..36c57de0a10a 100644 --- a/Documentation/ABI/testing/sysfs-block-zram +++ b/Documentation/ABI/testing/sysfs-block-zram @@ -22,14 +22,6 @@ Description: device. The reset operation frees all the memory associated with this device. -What: /sys/block/zram/max_comp_streams -Date: February 2014 -Contact: Sergey Senozhatsky -Description: - The max_comp_streams file is read-write and specifies the - number of backend's zcomp_strm compression streams (number of - concurrent compress operations). - What: /sys/block/zram/comp_algorithm Date: February 2014 Contact: Sergey Senozhatsky diff --git a/Documentation/admin-guide/blockdev/zram.rst b/Documentation/admin-guide/blockdev/zram.rst index 1576fb93f06c..9bdb30901a93 100644 --- a/Documentation/admin-guide/blockdev/zram.rst +++ b/Documentation/admin-guide/blockdev/zram.rst @@ -54,7 +54,7 @@ The list of possible return codes: If you use 'echo', the returned value is set by the 'echo' utility, and, in general case, something like:: - echo 3 > /sys/block/zram0/max_comp_streams + echo foo > /sys/block/zram0/comp_algorithm if [ $? -ne 0 ]; then handle_error fi @@ -73,21 +73,7 @@ This creates 4 devices: /dev/zram{0,1,2,3} num_devices parameter is optional and tells zram how many devices should be pre-created. Default: 1. -2) Set max number of compression streams -======================================== - -Regardless of the value passed to this attribute, ZRAM will always -allocate multiple compression streams - one per online CPU - thus -allowing several concurrent compression operations. The number of -allocated compression streams goes down when some of the CPUs -become offline. There is no single-compression-stream mode anymore, -unless you are running a UP system or have only 1 CPU online. - -To find out how many streams are currently available:: - - cat /sys/block/zram0/max_comp_streams - -3) Select compression algorithm +2) Select compression algorithm =============================== Using comp_algorithm device attribute one can see available and @@ -107,7 +93,7 @@ Examples:: For the time being, the `comp_algorithm` content shows only compression algorithms that are supported by zram. -4) Set compression algorithm parameters: Optional +3) Set compression algorithm parameters: Optional ================================================= Compression algorithms may support specific parameters which can be @@ -138,7 +124,7 @@ better the compression ratio, it even can take negatives values for some algorithms), for other algorithms `level` is acceleration level (the higher the value the lower the compression ratio). -5) Set Disksize +4) Set Disksize =============== Set disk size by writing the value to sysfs node 'disksize'. @@ -158,7 +144,7 @@ There is little point creating a zram of greater than twice the size of memory since we expect a 2:1 compression ratio. Note that zram uses about 0.1% of the size of the disk when not in use so a huge zram is wasteful. -6) Set memory limit: Optional +5) Set memory limit: Optional ============================= Set memory limit by writing the value to sysfs node 'mem_limit'. @@ -177,7 +163,7 @@ Examples:: # To disable memory limit echo 0 > /sys/block/zram0/mem_limit -7) Activate +6) Activate =========== :: @@ -188,7 +174,7 @@ Examples:: mkfs.ext4 /dev/zram1 mount /dev/zram1 /tmp -8) Add/remove zram devices +7) Add/remove zram devices ========================== zram provides a control interface, which enables dynamic (on-demand) device @@ -208,7 +194,7 @@ execute:: echo X > /sys/class/zram-control/hot_remove -9) Stats +8) Stats ======== Per-device statistics are exported as various nodes under /sys/block/zram/ @@ -228,8 +214,6 @@ mem_limit WO specifies the maximum amount of memory ZRAM can writeback_limit WO specifies the maximum amount of write IO zram can write out to backing device as 4KB unit writeback_limit_enable RW show and set writeback_limit feature -max_comp_streams RW the number of possible concurrent compress - operations comp_algorithm RW show and change the compression algorithm algorithm_params WO setup compression algorithm parameters compact WO trigger memory compaction @@ -310,7 +294,7 @@ a single line of text and contains the following stats separated by whitespace: Unit: 4K bytes ============== ============================================================= -10) Deactivate +9) Deactivate ============== :: @@ -318,7 +302,7 @@ a single line of text and contains the following stats separated by whitespace: swapoff /dev/zram0 umount /dev/zram1 -11) Reset +10) Reset ========= Write any positive value to 'reset' sysfs node:: diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index c1ecaac0b39f..d867e15d1d2e 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1103,27 +1103,6 @@ static void zram_debugfs_register(struct zram *zram) {}; static void zram_debugfs_unregister(struct zram *zram) {}; #endif -/* - * We switched to per-cpu streams and this attr is not needed anymore. - * However, we will keep it around for some time, because: - * a) we may revert per-cpu streams in the future - * b) it's visible to user space and we need to follow our 2 years - * retirement rule; but we already have a number of 'soon to be - * altered' attrs, so max_comp_streams need to wait for the next - * layoff cycle. - */ -static ssize_t max_comp_streams_show(struct device *dev, - struct device_attribute *attr, char *buf) -{ - return scnprintf(buf, PAGE_SIZE, "%d\n", num_online_cpus()); -} - -static ssize_t max_comp_streams_store(struct device *dev, - struct device_attribute *attr, const char *buf, size_t len) -{ - return len; -} - static void comp_algorithm_set(struct zram *zram, u32 prio, const char *alg) { /* Do not free statically defined compression algorithms */ @@ -2542,7 +2521,6 @@ static DEVICE_ATTR_WO(reset); static DEVICE_ATTR_WO(mem_limit); static DEVICE_ATTR_WO(mem_used_max); static DEVICE_ATTR_WO(idle); -static DEVICE_ATTR_RW(max_comp_streams); static DEVICE_ATTR_RW(comp_algorithm); #ifdef CONFIG_ZRAM_WRITEBACK static DEVICE_ATTR_RW(backing_dev); @@ -2564,7 +2542,6 @@ static struct attribute *zram_disk_attrs[] = { &dev_attr_mem_limit.attr, &dev_attr_mem_used_max.attr, &dev_attr_idle.attr, - &dev_attr_max_comp_streams.attr, &dev_attr_comp_algorithm.attr, #ifdef CONFIG_ZRAM_WRITEBACK &dev_attr_backing_dev.attr, From patchwork Mon Jan 27 07:29:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13951025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9D76C0218C for ; Mon, 27 Jan 2025 07:30:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3BB4E280125; Mon, 27 Jan 2025 02:30:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 36A672800DA; Mon, 27 Jan 2025 02:30:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20C08280125; Mon, 27 Jan 2025 02:30:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id F3F882800DA for ; Mon, 27 Jan 2025 02:30:08 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 84118C0646 for ; Mon, 27 Jan 2025 07:30:08 +0000 (UTC) X-FDA: 83052408096.25.DC7D96D Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf11.hostedemail.com (Postfix) with ESMTP id B1DA040014 for ; Mon, 27 Jan 2025 07:30:06 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=jK1Yf0Cg; spf=pass (imf11.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.170 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737963006; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QR/p5Tl6lRpVatp+KkNy0La/JT3xx5EKDzU3nlrXbQA=; b=E3mUKwuZtEevtBWELPH6H12Zfl7y/H2DUWvkTxsOU2uG46HCbxitf/nfbLLecrlr07yKCE 2kfi6RuxSyfjk6sBHSgK61/2pH0ph2FMYy5vckBoAjys3N4iB5eFezUn8uCKdm31aqnqbO 3uZDeiR6AIEyW42ZVvWCgqbCKGKcgUA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737963006; a=rsa-sha256; cv=none; b=tViJR0gA3QinUAZ/pjEKcFQsZWbyWTAVtmJ+prCNvn0ptVqlCq1XsLO/tPRtIIzrww1pxt mkQeANzRXyvcz+UutdY7pTuLtEYQ66QMztO9Vn/V0ljN9WoLwX8rdic1bgNbLkegkDs5fU +e6T95nIpWDAIZCaIfVmV0h+KX//xWM= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=jK1Yf0Cg; spf=pass (imf11.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.170 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-21675fd60feso90566205ad.2 for ; Sun, 26 Jan 2025 23:30:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1737963005; x=1738567805; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QR/p5Tl6lRpVatp+KkNy0La/JT3xx5EKDzU3nlrXbQA=; b=jK1Yf0CgpQp4dNQgst6JjqB9bl0i24P7n10IpNzE5OsrRlMNTGgIkhj6Etmenv+g37 //nwkyyApkh5ZuvjWbwc6satfod1i3Ohi+AsMtP2sYfxF2zNnvGOmkYjcWvYiONi6AQk qPLafgAQafE7LWM0p4tMWTP4mWKRcODIisNxs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737963005; x=1738567805; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QR/p5Tl6lRpVatp+KkNy0La/JT3xx5EKDzU3nlrXbQA=; b=YnmsTIxNglcJnRh4XsGw3aWJi2qrfPfixO+JWCIDEERqzHG9BXDzFesLs4qRL65NRy zNJa7Fns6BAlGWlKENYDo2aUNAYmfkOv0BGIAVPaOr0hmOCXL/KuLnrVyrHafG4QWNaO 06iocrR50S3J2EuslGGYVsWIaUImXcB62xhtS0MkBXo9wzpLtJ//TujRPyV5C5jVZmbs XOVygDlvSaAL0M2sFrziRnvoH6TQu9DFegfUG8AfxUj8kQbs4jZfL1Zke7+SajimvKdc f1Y4kZeHxJTcf7lCBZPjOR7Aqp7QwXL/PnNLpfWODocAR1c6Ky56YN67cNdI5EKqCA9L ZWFw== X-Gm-Message-State: AOJu0YyQP2jZ73ONI6CM47IWHTB4xrZr5e0M2QdFHjdKshjRzZuDQTVd u2XO6lwfPmvjWgpeAGPhxuCqHGxLmG44WwWncp+i2KHlv4OGmVSPpivLhuzGrg== X-Gm-Gg: ASbGncvz9oe5YGXdMtD9wo00E4jvs1eREi4PNXVznmffG7lrkRzaAExD7nl3fT0AYBA nzsGuXrFWun2Wt/aQW4ru/pvfhJ3eMJHjqa0xkd+CX8LL2pPyhWoo+Y/+JjBaoAk9E58jmHvdlU Xf77hZT+i1nJCdqSE4gkAR8sIX+Q67hpQd32AsF+Qvo3ASKCfwQGmjP4TBiandigqTbvM2ndzg6 1BP0HB9LLAZO4aHbZHgwGlkrbZYHHRLcgZvLMf+Lkf/feJ0lOP3Ssl5m943A2MnJ/6+3dH39eTk QeRv6AQ= X-Google-Smtp-Source: AGHT+IHV7YMpzbIT6L+DxKKXMn+vhmqmbhrzw8POAb9AfcZ1TJR/LzmJE93baxj7ELaSGxd4dqLq6A== X-Received: by 2002:a05:6a20:8428:b0:1e1:9f57:eaaf with SMTP id adf61e73a8af0-1eb2144d54amr59526793637.6.1737963005514; Sun, 26 Jan 2025 23:30:05 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:566d:6152:c049:8d3a]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72f8a760e1fsm6352930b3a.113.2025.01.26.23.30.03 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 26 Jan 2025 23:30:05 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 5/9] zram: remove two-staged handle allocation Date: Mon, 27 Jan 2025 16:29:16 +0900 Message-ID: <20250127072932.1289973-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250127072932.1289973-1-senozhatsky@chromium.org> References: <20250127072932.1289973-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: B1DA040014 X-Stat-Signature: fogrkqpwypjfh3wpxk7u1z1b4hqxgzkw X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1737963006-364241 X-HE-Meta: U2FsdGVkX1+yENMoH/pZkHz8/T1G6NMRd3M+viDGP8EAw7FE3BjQjyreKpHOsrWv8pkKzOa6arjS5JzbTjiaggp2QpSnInmYa47xzQDhhvav/AxFG5UsT3uk6EMWIVYrB98Unwi/U3fM3o/NUxE2feWU5Af8Axyr3Rd/NVdaEg4/MeiEmBAIUAFzahkRghs6WxSG6htf864OYTF8ookoEmprE9HdR4asHk0IqP1J5KSu7xYG6zEmgE4mAyV4YYCQhfX17y7VSM6+mCWdYJDiNk09efxFdr+YmskX77LTRJTxKpx/VKrya6MTCTmjkwCr+zrolMGoNI2f6VpBLA0m0Jmf1BXRVDTLqmgYHpF3vP0KRrL3kx81/EORMuMSlqpMCjULg2byP8nKN02ZOzHKRiddyQUF9U++tXB9fh/yArKlMnZNo6wG3JAQ2yHlnNn409k6m7tyFcxUle8/QvnCIhjaLVidCDlCfluFJxaYhE0jTldrtwaHWyAJJvX4luGK68NuBiOEmr87ezqpkfIP5qkwiT01abNJ+i2So2IOFfmPiBHV8vxjLy5BHalpgSONm5qjQ52zW3f3YO5r5ShK3DtySif/FGS0RqR1OVPrgiQrTWY/dzyOHqeZw1Of6sbxw5cUhQ+54O+Y4zMqIlciUdJnzTylPqfpUy/bMI+Mai+FR+UXi3mBmTOlbWxVSphNXeZ3/YhpvkOyRuC7E9cI6NggCV3pi9wYncR/PPKsn53Bq7ZGqNdez8q/DWMBkj0Wkx3FLJebXyrcXLPDp70dfBdlbUNsu8cZ1kXFK+hA+ovs8R1Xe0NeUz4u7V0xqbz7f0SfjhO7Opm+pjEQawy27lbzqJAwnED5b6hRndKJ+5y546Ge7AfBgT8ff6dUM4cw0wrKp+LnRyJKWfYjLoDPLpPqnvSjiFMYx3T4goDyFhTMfJ7/+iAPV7I5KaRiLMPaxIIl3jB5DsLzIusNFzz 663qR50H UohUTOvuaaELB2YQ0RlmqjdT9OlD1eaVnDiPapNsZikou96wn3AKub7eDNYO1qYRiyjgdlE3dSHXSDHVqLJfar9aPhZNqJ+kuw0vrp0A8EtCervoyctCOtaRcQRezesQK2+XObQLpRYIAnHHIPOcdaVKygdmaH8SKEszEm6wI2cqRWHgfwDShjopHxJC4MokDUXuX2hsOMoN3BrF2r8MOX8sctQVaxjq22TgFUW7d44JxXSIAy8AGJPQW5WMj4tJELDKeE58REVYQfsG+JhQvRbg3HJs7Jp/2bPJIYn2Ns8oldY3YnIhNWxo/LA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Previously zram write() was atomic which required us to pass __GFP_KSWAPD_RECLAIM to zsmalloc handle allocation on a fast path and attempt a slow path allocation (with recompression) when the fast path failed. Since it's not atomic anymore we can permit direct reclaim during allocation, and remove fast allocation path and, also, drop the recompression path (which should reduce CPU/battery usage). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 41 +++++------------------------------ 1 file changed, 6 insertions(+), 35 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index d867e15d1d2e..046c80aa4310 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1725,11 +1725,11 @@ static int write_incompressible_page(struct zram *zram, struct page *page, static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret = 0; - unsigned long handle = -ENOMEM; - unsigned int comp_len = 0; + unsigned long handle; + unsigned int comp_len; void *dst, *mem; struct zcomp_strm *zstrm; - unsigned long element = 0; + unsigned long element; bool same_filled; /* First, free memory allocated to this slot (if any) */ @@ -1743,7 +1743,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (same_filled) return write_same_filled_page(zram, element, index); -compress_again: zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); mem = kmap_local_page(page); ret = zcomp_compress(zram->comps[ZRAM_PRIMARY_COMP], zstrm, @@ -1752,8 +1751,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (unlikely(ret)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); - pr_err("Compression failed! err=%d\n", ret); - zs_free(zram->mem_pool, handle); return ret; } @@ -1762,36 +1759,10 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return write_incompressible_page(zram, page, index); } - /* - * handle allocation has 2 paths: - * a) fast path is executed with preemption disabled (for - * per-cpu streams) and has __GFP_DIRECT_RECLAIM bit clear, - * since we can't sleep; - * b) slow path enables preemption and attempts to allocate - * the page with __GFP_DIRECT_RECLAIM bit set. we have to - * put per-cpu compression stream and, thus, to re-do - * the compression once handle is allocated. - * - * if we have a 'non-null' handle here then we are coming - * from the slow path and handle has already been allocated. - */ + handle = zs_malloc(zram->mem_pool, comp_len, + GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) - handle = zs_malloc(zram->mem_pool, comp_len, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); - atomic64_inc(&zram->stats.writestall); - handle = zs_malloc(zram->mem_pool, comp_len, - GFP_NOIO | __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) - return PTR_ERR((void *)handle); - - goto compress_again; - } + return PTR_ERR((void *)handle); if (!zram_can_store_page(zram)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); From patchwork Mon Jan 27 07:29:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13951026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C96FAC0218C for ; Mon, 27 Jan 2025 07:30:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 62B67280126; Mon, 27 Jan 2025 02:30:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5DB8D2800DA; Mon, 27 Jan 2025 02:30:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A33F280126; Mon, 27 Jan 2025 02:30:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 28CD72800DA for ; Mon, 27 Jan 2025 02:30:14 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A775DB0E54 for ; Mon, 27 Jan 2025 07:30:13 +0000 (UTC) X-FDA: 83052408306.15.707B58B Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf03.hostedemail.com (Postfix) with ESMTP id C8E3B2000C for ; Mon, 27 Jan 2025 07:30:11 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=fwMeiESF; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf03.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.170 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737963011; a=rsa-sha256; cv=none; b=sKlSuipS+9hy/XUl7nLNju9Rm7p/SbeybQJDAbCLtLBg86OmZq+Xf6bT/hjDGNn5elg9A6 RsCQS1V3rwqlG9u9M6DHpo6UMcnKBIxnha7nvfQA1UzeT/5agpSjxSGw2UwQDmJvHhYktN Qwy7giMEzd3v9nRmjgWM5e3L93Hu+2I= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=fwMeiESF; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf03.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.170 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737963011; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5g614UtqxTvOkBYeQdwAP9Uo/j0Alc44lQaUMvbAkkk=; b=QnZQAOnim72nlQpxKedj3juvwSwei2r2D3ZTwmNwcFGi7m2NxM5DsiKqj1W1u0GWN7Q5kj 3UG6E2LS7tLALNhE65grtOTMTkmIXVe6AqMYT7LYM9GslwUrDS4bi6Lyb6VEqtzSgUVH00 xlFXZagGfRUpjYlYTpMhrBt3ItC3xdA= Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-21628b3fe7dso69334405ad.3 for ; Sun, 26 Jan 2025 23:30:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1737963010; x=1738567810; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5g614UtqxTvOkBYeQdwAP9Uo/j0Alc44lQaUMvbAkkk=; b=fwMeiESF76m6mNMa6N2kxZNv6QCyJD3mghbRmBwBDeehBHYVXK83adyAs5dgKxWt+l qQEC3SByrd/m4xIvd6yLRhb9n2VwBSD07hbjIHTZKpsXadFPc1b1TeR9Jga0FoUkFIiR vVpwoTYyMc4wwHGylqJIS9JJIRnqLAcuNgYRs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737963010; x=1738567810; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5g614UtqxTvOkBYeQdwAP9Uo/j0Alc44lQaUMvbAkkk=; b=vmFqtBoJGdpC7RTZ8v0YiMceiUQM4a552qfE6M56XGpukj4LtOHx0HFYx6OyRygOQK PBiaLAzOHpVpkKpfc7LGShX4xG6LtoqYtkO+mfwmAur2rhjxkYy8mVjjY5ufaxBwcdDP 6An/PwLOxSHPa/ip4hAYJSn8YNChs6STrIj/enCmEZMibJYSlwdJLOKg4iQH9IEXPBWE Xt5O8VYh/+sre7GsPr0WYQavEZc9sTGo17ARMd3v2N4Y+NuzQNWgNaxnsEJR49Ust01I Nj3Nq91/H1RF3IZ/je6yCQnSqFvjJ1ZjbC39toa2Iyoaqbj2Lk76RTnjx55CRZh9tywr B66A== X-Gm-Message-State: AOJu0YymJV8+XGgKtd6cCpVB+2xrmuIc4ziNQ0NOOJEUiKE180cqtaib CuZXqbnctcKUVJ1c0PhV0iAD6RivWFIxBXTggtpEKE5VtIBtsQsYZUqu2SB9/A== X-Gm-Gg: ASbGnculCJYgxN06Etcurk90KovtfzHT3gTskgbNozlh+tURI0OIT2QVU4HrxozYRtY TnesU5rWf0NQIe6frmm+GwB6Alq8u7TDh13jbt76hMi2MWV97WWXqMJhRHNqo5v23cbzZKBav8Y 1Q7WfLzVKnYgsBfI1fCIHpzqfE2TAzlbGamF5fD+D/KKTlZTzExSIYQJP0xv9h4J1HXhg7K9hIq yEfiC/4/T4aCcWMTbIMmrN7uDmXuNyoZG70XP9whGQ+fCgCK9UjQ87OL/9IJDg47niCaAira+ud YrpKmw0= X-Google-Smtp-Source: AGHT+IGKyeWS3mv+loODSXHq45Ym+6tplx4pvP6lVRL0eZv4C+tLZFBWcDTabbbQlUIiu02lYNkogw== X-Received: by 2002:a05:6a00:2294:b0:728:e745:23cd with SMTP id d2e1a72fcca58-72daf92bbd7mr61064828b3a.3.1737963010417; Sun, 26 Jan 2025 23:30:10 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:566d:6152:c049:8d3a]) by smtp.gmail.com with UTF8SMTPSA id 41be03b00d2f7-ac48ea371dasm5724120a12.12.2025.01.26.23.30.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 26 Jan 2025 23:30:10 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 6/9] zram: permit reclaim in zstd custom allocator Date: Mon, 27 Jan 2025 16:29:17 +0900 Message-ID: <20250127072932.1289973-7-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250127072932.1289973-1-senozhatsky@chromium.org> References: <20250127072932.1289973-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: C8E3B2000C X-Stat-Signature: weraamm4niof3fri675n68ubnpw9ba8e X-Rspam-User: X-HE-Tag: 1737963011-221895 X-HE-Meta: U2FsdGVkX1+3iWcTR7494axOdxcppL+6qtHq5gLqckeuNbnMbDdE9oQfo3FhLZWDpVCjj0OrIKiOku9m8EjP7B0ygbRDl3Mj4dmqXoF1bquqZl9nfuFTMi3CBWTx/D5s0ZdxW3aF/zewKw5udWtBOIjzZsnfKBg2DixgNrlRy2R6qFEyoUS/A1ZZXCfh8qZircavWQhrdxmoo6uEjQ3G671iIjnGTziDTodRloxmJxFN3Dd7shKHiHv2DFsZwNzlmAPD3KG5liq9XYC0JyCZQTvIz2cdYHxbL4MBPCFpIWvQTVzoKEwg3qa5DgkwUZyUlaa2BNN7C0Lpfd/oju0rVk+mfnd7aK22n8sQgJTOUJ9936hHph1z7FvcEKM3hOvISy48qtFkUbVj1lPRfr+T7BeoLTxeK66yLCWspGjmtn2bwxji//sS/hap3DcUu0teAv1oS8da96Z6NOI8UeNCsC1dk0NE342lH88kb4rxlVoe2EYYt1o0MiFz1kxNYbDQIQ1DX0/jhv2ctnFFNzU4hp1oZaA89EKDpu3ysKwgalZzuqZVJvFd+hS5ez5jS67Vhcp0qWpgemnc7bf7h/WybsdsQlMX5v5Dv7imf6xHBlxBppK2T5dgHa4D6PlCrJ6KmwD2VPVErcWtDviQlcS8ptYRFViVVu3GpmswAbB5Loji1o0znU8aC54xb7FlzzvJs3uh4VyvNmeJlJRYfOoxO0S2kMqR/RD1cpNCkGDcw5P6cKzvdU3w/GOGSCF+AlLgoJM/g7EHdXHYipl/pnYEZ+EJlo17OK8uB8o9D3GMvqEa1/NS4RvS4jISWWwrvp9Tq9iyGxyG/G2uC9TjpydzMaKZu1QfD8MxgcfX9VGGPB3S5cmzZ94C7wrMdKymJ1dCwxJx3r+loa9mBTb4kMAZwcJd6rL5vZYnO/2sPSXSEd2Mxjrs0dZuLCKfGy3NZkcRqzcrzrIBduaAAa5CznW xEvFw1M2 P8BGvIxtceBsp8ORP3K5Mx5HgSvRbS5AlxYz+J9Wxb+X0p7zBxItBVJCVbmHgitpGELUzB+El6Oby0eEEU9syjaMheU5LgboJkYFFb7uWSxPF7vsC96WjvSvPSwH4DPebucD/C2PzGtmMLblbnoiyhV19opSYhULGoW/ifsdt55nRZyZtQ6v58HlZ8QHTuHINdl1r0qifrz40ll2X5Lk3I7ih/EmcOtbvx+8im0yetTXo2kmPtNQhyZAi17hMRnHUwnVfXeDAAok2cXJ5cJuWx4yT7AlHU6cf/nzJsj472jCPqWdShOj8lqVTzg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When configured with pre-trained compression/decompression dictionary support, zstd requires custom memory allocator, which it calls internally from compression()/decompression() routines. This was a tad problematic, because that would mean allocation from atomic context (either under entry spin-lock, or per-CPU local-lock or both). Now, with non-atomic zram write(), those limitations are relaxed and we can allow direct and indirect reclaim during allocations. The tricky part is zram read() path, which is still atomic in one particular case (read_compressed_page()), due to zsmalloc handling of object mapping. However, in zram in order to read() something one has to write() it first, and write() is when zstd allocates required internal state memory, and write() path is non-atomic. Because of this write() allocation, in theory, zstd should not call its allocator from the atomic read() path. Keep the non-preemptible branch, just in case if zstd allocates memory from read(), but WARN_ON_ONCE() if it happens. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/backend_zstd.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/drivers/block/zram/backend_zstd.c b/drivers/block/zram/backend_zstd.c index 1184c0036f44..53431251ea62 100644 --- a/drivers/block/zram/backend_zstd.c +++ b/drivers/block/zram/backend_zstd.c @@ -24,19 +24,14 @@ struct zstd_params { /* * For C/D dictionaries we need to provide zstd with zstd_custom_mem, * which zstd uses internally to allocate/free memory when needed. - * - * This means that allocator.customAlloc() can be called from zcomp_compress() - * under local-lock (per-CPU compression stream), in which case we must use - * GFP_ATOMIC. - * - * Another complication here is that we can be configured as a swap device. */ static void *zstd_custom_alloc(void *opaque, size_t size) { - if (!preemptible()) + /* Technically this should not happen */ + if (WARN_ON_ONCE(!preemptible())) return kvzalloc(size, GFP_ATOMIC); - return kvzalloc(size, __GFP_KSWAPD_RECLAIM | __GFP_NOWARN); + return kvzalloc(size, GFP_NOIO | __GFP_NOWARN); } static void zstd_custom_free(void *opaque, void *address) From patchwork Mon Jan 27 07:29:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13951027 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0B39C0218C for ; Mon, 27 Jan 2025 07:30:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3E3CE280127; Mon, 27 Jan 2025 02:30:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 392062800DA; Mon, 27 Jan 2025 02:30:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 259BF280127; Mon, 27 Jan 2025 02:30:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 03CA22800DA for ; Mon, 27 Jan 2025 02:30:18 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BB06A1C9BB1 for ; Mon, 27 Jan 2025 07:30:18 +0000 (UTC) X-FDA: 83052408516.18.F4C0C31 Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) by imf29.hostedemail.com (Postfix) with ESMTP id E0BEE120007 for ; Mon, 27 Jan 2025 07:30:16 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=ZhXlukGP; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf29.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.49 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737963017; a=rsa-sha256; cv=none; b=emZLRaHGtIbkZx3QHEJCRDCM53U3ObFBsjkKnajf8NBn2v6yO2V+z/TtXtZjCeJP//WbV9 mtNY1GLO6uIlJXCcBN2OMEl52fA2Ug4UIH93fXQW8I5QSkKYVpb4GyNowiRIppaIpzGgPg HeSpYt5BWUp9GaaTh/39LP+XR1x6/Fc= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=ZhXlukGP; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf29.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.49 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737963017; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eEuN8Bq5p1mqZ6qTtfa1iRJPgB+Yv5kUdAyDKcVHFeU=; b=k5OhwCjswH/bOxb8mnFcpx5+3rLCHuMZmvZrElEEG8z9eg/9/TD4TakQG0IGghYhuXd1kx XXnue54pQvnGKVgVETnn4vBFBYtQj8tbdBvuzUb+Dy6xShEmybzt97j/HweHKOlpv9bUKH 4GBLoZBsXdq7c/5+Ig5xPeL9Re4pz4E= Received: by mail-pj1-f49.google.com with SMTP id 98e67ed59e1d1-2ef70c7efa5so5601710a91.2 for ; Sun, 26 Jan 2025 23:30:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1737963016; x=1738567816; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eEuN8Bq5p1mqZ6qTtfa1iRJPgB+Yv5kUdAyDKcVHFeU=; b=ZhXlukGPBpGTnCOD0mU92LWFp/ytN+HYeWHeREAjclbQMnA6xrVGifcTA7wPOYkVtg vc2II+Of7AlmGeTpZ5+MAg3VduuCm7Xi2BEjWoMlGPQbc6frCZIJjv75UNoRVQ1ke56G XCtV1x/OxaE2Cs8sGvOJTtsDYoSuJTM1mg62I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737963016; x=1738567816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eEuN8Bq5p1mqZ6qTtfa1iRJPgB+Yv5kUdAyDKcVHFeU=; b=AXhCgdGP1B+P3kGa0GwqPkvyg70D26ZCr/O5W5PK/NFMUtgy3g2KAGMcUrzue8a14J tQqBJM7zqSlJTgpX/Y2Px8YbGUsOn8D5PvT9UVr/gCeaDhqLzl77cOBhW9ecs//VOW0d JF4IkDkZyy+4SU6B6JFQF20JpldqTiNzGLbBEsjKix9/6hqMkMTY/S1DLLdPraWSrsCC fZdMbdCc5Qwek96JmTBzkXDOK0nXS9i6x/PBPHvP8Ct/MNopqrb2x+nXAdagHC3GmylB 41ok5ueMbjYGeZOjG3e6Ow6WAYWoEJ8CzcYd2hM/IGpC+1N8It72rcWGe41wIr2aKjnf MUdA== X-Gm-Message-State: AOJu0Yz/U+lTJo39a08otwXgsNbfC3pwx94P2pCeKpNLvPaPcmj35puZ q+NZPZ0nKETxTkQrrvuuVPMTdE9TuXvE7vAULrcPmmLInmyd8/n75HCPg5VjfQ== X-Gm-Gg: ASbGncsJAIIxE0dshrpX6/btQ43R/YRZzym1WQnzASPqhkjiJzwxkzOCvYOcywRLFyD pKLK4Rhm4PoaejHTelzxx94NdiMkfTOjQcZENyQKFpxh3OyNuRODk9fMRLiAFgGery+3jDqoEEF okeHfZqbJjiOiTndFrQoY4MRL+lfxwMA0tUTpXukkfygEhhvSVcdeN8q9DEPwb3+lKXHTB6I1ZS Wd3k3g6ddWtVLcnroAV8G0Ds6pWJVUGRdhREEHZpaHaL9nZ9Dl+vjURp7ZTFQbm0kuqQwng5Nkq nsy1h8E= X-Google-Smtp-Source: AGHT+IGlo/vTdElSIeoYoZL5xyWIf44NgB7/wcjQQDIBMbG82N6RjyywPPzveq1A/4IFqrAaMweY1g== X-Received: by 2002:a17:90b:5249:b0:2ee:e518:c1cb with SMTP id 98e67ed59e1d1-2f782c4bdd2mr61051697a91.7.1737963015607; Sun, 26 Jan 2025 23:30:15 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:566d:6152:c049:8d3a]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2f7ffa6a755sm7123997a91.26.2025.01.26.23.30.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 26 Jan 2025 23:30:15 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 7/9] zram: permit reclaim in recompression handle allocation Date: Mon, 27 Jan 2025 16:29:18 +0900 Message-ID: <20250127072932.1289973-8-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250127072932.1289973-1-senozhatsky@chromium.org> References: <20250127072932.1289973-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: E0BEE120007 X-Stat-Signature: yp6a3h9cjutqj73xpm5u7pux14h1qout X-HE-Tag: 1737963016-680993 X-HE-Meta: U2FsdGVkX19XCxne+Djg35xykPuGjqfaPXvprcyZjtTx3AXksYznVRAKbcb63amW9EjIxX71lvs1h8XFEszzeNm4wnUWFoJjHTMkkEuuRLCCqLKsonZOljKBLNhvvuxcgY5GA/vrU2Q7LIi0UDwAcWsR01O6jPEIYDIvSXi/yicFx5mtt++2ejMH6vhdyEWOedCkVVsJhvg6yJQNPFeSTCsiXlfpXlXF+lFZXHQGmeiVoswUnnKTfffgMT5jwMYnVhPStFO+Gbh1a0fP5SUpTx+vxJG+BtePMUkZz0aVUFW3bkv/O0lklrAEFUNL6dm3l4bQGwzAnzjpwZp9WRE542sVCLnKKuUNiG9bduMNjs96e1fDLwejvPgHJXEHUCoOlOcHJXXpzi4vqrKs/VmCJsR8OhI0mhQ0DOntianK8N6sWSrQ8wUlmj1G+JXVDYyS7zmU6Aa1h4OkPkekWAuOLP3uDmoc4IqKNERLTrVZONJtPwFTdL3j+z3Q7Kq+kJnk66dJKaQwjUnQ2r+sGhoXm66Yi6L9+/QcNV2T+uD+540QBb3TERpfmTBJiVDb3kgYIoSGUx9m65uO7v84F0Rml1zO1mAUbK7d8WyaHpCWwkP5KpqRXA/6pNON6SWQG8jiXPsHRWfaRHXxipqEv9WKmXsM93k2RTqyv7idtnZbqVkoeT5dnkD1xeKSvWzU2SKDCSPoSCIdmLQ6Rk7BG/6PaOzlTAqnjbbj7geA0dYhIeIPIQyivKcOktzVetcQxxq6mjyktQRR0QIbr8pqpvi75+FS9QiwdZxk8NENCMFd/jyWMGd9IBEU+0XZ3QfoQt8lPVOighSnzePOAcPNpMiPj7OLMLgb3L7LUtLMu2QDgkW7i5nL/WXZvC0zmbzvc/wE4Zs5hsIrxHzeUU+M9O7aPVMMKEl64Pbmx42W8pw7fmeNBj1BylILsDrVq5n6J7H/A3PzC1hfJr9PFbakJXI MNAXVnyf KdIBzojRfst0fF563+mzpxp94vII36mR2/emcyMECSH+Rc1LmjR2GTCTVzEcfU6ERmw/JR9a0yTJRL0S3x/8nSkuKR4Dlf1QloQrcD9ED94utui8m98M1tno/loZQM7pFm7sr8EL6Sh0QEiZFOQseGndWtn/BYrL9F+441omgV2jhm+fQQUU+na4MJO/02AJlVfVTe2U+Xkty+sqTX/5ujHHWXqYYfSnM3BfZFCHpOqac9UAc4VFLiZtQ89oyUjgLW0dnZxT/r9dWj4b+mbUI6e40yptqsBXWC7bzMUgBbNp3tKzw33qrTG3Z4Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Recompression path can now permit direct reclaim during new zs_handle allocation, because it's not atomic anymore. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 046c80aa4310..10e4c6a7735c 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1988,17 +1988,11 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, return 0; /* - * No direct reclaim (slow path) for handle allocation and no - * re-compression attempt (unlike in zram_write_bvec()) since - * we already have stored that object in zsmalloc. If we cannot - * alloc memory for recompressed object then we bail out and - * simply keep the old (existing) object in zsmalloc. + * If we cannot alloc memory for recompressed object then we bail out + * and simply keep the old (existing) object in zsmalloc. */ handle_new = zs_malloc(zram->mem_pool, comp_len_new, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zram->comps[prio], zstrm); return PTR_ERR((void *)handle_new); From patchwork Mon Jan 27 07:29:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13951028 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A68CC0218C for ; Mon, 27 Jan 2025 07:30:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A9981280128; Mon, 27 Jan 2025 02:30:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A48292800DA; Mon, 27 Jan 2025 02:30:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8E999280128; Mon, 27 Jan 2025 02:30:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 726482800DA for ; Mon, 27 Jan 2025 02:30:25 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 2ACD512200C for ; Mon, 27 Jan 2025 07:30:25 +0000 (UTC) X-FDA: 83052408810.28.4D667F4 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf07.hostedemail.com (Postfix) with ESMTP id 53B434000D for ; Mon, 27 Jan 2025 07:30:23 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=SfEzD7IJ; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf07.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.182 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737963023; a=rsa-sha256; cv=none; b=HCKePmr8hv/FzY+8CuZUwGFLSq6Bz7VPJ3AO75OaclmP0AzFbC/2neoKpEDeUtw0MeTJzk 1wJbSLXMBG3QPq1tnzWBlynZXXQrLguig4SjcHcgGsM2JXquwS2OiW9toNSPsvFEyUyVRT jCF/Ezzi+QPcDEQtp7F5PjN+kn6zDNA= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=SfEzD7IJ; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf07.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.182 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737963023; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kSgdSt89nTEWoMlDHQC4zKaDJjnbZfjCeF+nCuROQ6c=; b=wndLDVUxjES+FdSAq2TpEmLZa6MKb3BTNDIViMVxkck3rNfgc8NOfURiNFnurVQWesTK76 OIW7GAuuzQ0/q+CGm4h+tm1EmzJgNj5L6oW4yWpGWayKMnUuKWaNhZw7JzAig8ROWicUV2 atD6pn+XmVUMUH5sw9RKVSqTnV4MCCQ= Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-2164b1f05caso68078395ad.3 for ; Sun, 26 Jan 2025 23:30:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1737963022; x=1738567822; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kSgdSt89nTEWoMlDHQC4zKaDJjnbZfjCeF+nCuROQ6c=; b=SfEzD7IJKwYIATMCTkJX7+qV0H6yKKrwN/0qwt88cGCJo4mNYd6UDVVjtyKxIqt+Pp rpMeRRDC4Eu+Vjbu3UjdhhcIHFf9x8Dhmw6l1qe5fcIxikg3W5/zJeNVwmbg66E4Bv9W 5RC7ukmNee4Xxd5RukIHjhjhfvsJWygP6tY9M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737963022; x=1738567822; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kSgdSt89nTEWoMlDHQC4zKaDJjnbZfjCeF+nCuROQ6c=; b=mhB9Y+Ncukp9DHgwRcCuTYXXtwC3zvd69AJBHPRp85q684r89A6YuaBgAy92vIuXt1 XdJxWmlfsUWo0YXPpkC8VlvZhXmFsjhXB/2zWmlcXPDIRpd+TPrlN9W602bhI46ENLl7 3fMI8qeZ+48PQIJz4iLJyiT3glowhuPH1OyaObUi3alEzdhxRuIRuOTrIwsssQscJIZ/ 5k+jE4EYp/hMElwOWMl/DR0pfnoAX16j5rOXj8KLodQPICjMVXJ+g1VdaCDu88CNc8q3 9UQTE/ci9nYVbLPiKWSsumfQm2268zv7hSb4BFCbg84KMbQp5MLBCIg4akp4YdLjSMQM A0Fg== X-Gm-Message-State: AOJu0YwtLFB0zOrGJKeRGxrA2qGBPIVDax7Fm+zkBKCvDtpfjvZpRt3b fnGELBSPRmbkgku0lovno4dx3nwpuInUylaMThpauxD6g1ZUdch8rfUJ2y4GtQ== X-Gm-Gg: ASbGncuCUv1YxnIk0z3cOfwBKlENeICTjl0rpOt323ikKPf2zGOgF+/UQJiCUnh/rpa BklBRUIVrHefqfOSxMHVZzAHdN3FaSps/uosZ/nIGvMuK+YQJ/fRavDJajW09Tn4NimJNOyAZjA KTnVUpGyySxY/1Nrgz58AeWIIM22xBjW3HG0SaH/OCKr1YOICebkkKZwAGaqNDsNckOOARb7aEl 1SvE1leta1xb412RgKudUElcAHt5SlGWogkZS3Z6ec4998o09U6lTvUJnvyF9jvKrZ+fnCcKDYx DYoerjc= X-Google-Smtp-Source: AGHT+IFJkNEfxxGm9e2mhZinKXdaG+6TaYhwskg1qiYEq63JEXJ/MVr0HkgBllV11tnfS2o+XZnVNQ== X-Received: by 2002:a17:902:cf0b:b0:216:2af7:a2a3 with SMTP id d9443c01a7336-21c3560726cmr685117125ad.53.1737963022243; Sun, 26 Jan 2025 23:30:22 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:566d:6152:c049:8d3a]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-21da3d9e092sm56864765ad.28.2025.01.26.23.30.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 26 Jan 2025 23:30:21 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 8/9] zram: remove writestall zram_stats member Date: Mon, 27 Jan 2025 16:29:19 +0900 Message-ID: <20250127072932.1289973-9-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250127072932.1289973-1-senozhatsky@chromium.org> References: <20250127072932.1289973-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 53B434000D X-Stat-Signature: fccmux5mxnrd9jyr4k7kzuccz7rc4bx9 X-Rspam-User: X-HE-Tag: 1737963023-111677 X-HE-Meta: U2FsdGVkX1/SE7k78DTnZuwBRSBm8bd58OogyyBtVSgKyWQo8OV6hFne2vz9OuA4pGROmRVNK3cFoGJFHLEa7SQdkhhZT+V1rbUnq+VJ0spCUqtlZAEnDt+eEb9W4M9kyt9AAcZYGQp8NszIRlmESl2+/lM+jHru3HpaEG86qZxTfAMX6Q4VdedU0KJNJ7r4vt5T1bw7GOsWBbuCOcmbEDvmdUHPjbs0Qs2CsCVscHvBSvzBIihLGaOtJu5EV9oFhgBOTOPoW8oaLZwDfGYPvbDrz/APIyABuaGXOZmi2MWGww8LpGGKE6wxvZo1/RaPBbV0fr2ZDX/WaUeon2IaYApXkTXnBZvHAE0L3N84jXuD/zMUjjWIaK8haj/a5ryEvgwCPtLIXR48NCixzBrCexFu97zRD4JkCgOHgrAFORAUn3/E8BJ5w7yO8ViOhZ3TuHu1jhZq9mCzTROl3dxQ6rxtlw5K4S6KgVJMj98nmR29iwlC8oTEsEvcOjyBnnpNbUWmtfAFOZXMC842XpNE2Z8q8BM2ORE/4xVQVdN0YRI7QnlyCtslaDTDX0kh3dZui8lmP9eYhQpY1QhmW0Uoe/m4s4U7KILDd3laMQDi+gtb+uPD4g4pxVh3ytRqtubPiarIhLZVRBMr90PxPiHNomycDAjWyVK1rJ2gCqfXCOR/Bm9+5wSAgOxj4NgYuit+E726j0fTj574CSXSkMgrcGJJNwVpNBgUyXllOrVJslz30Le+QR/Crhs4kDyb67hFY5Mlfvoz1jCvAhn70Mnp9vMe0UZzkNpdKrYSOr0cb984w6ndV7BRJc2byjYUArE7VHTwmW+5aE9p/Yg+n/U7hVtfBUbEtq/7ojK9LbexDi/27hMrEoc3i9JAKHJoKz/8PeAeZsDn42rLO7R9Txftdvu7JZD9/bdbcoT/99T0shC4eUuhmgpF6ppL/tkiBn/792iKg3YLgs1LeiNX25/ lL9FyC5b 4Xv0FloLzjc7N2DepRSqCq8H9KpDLcgoGbm4MWopT/avOp8tSlYlzCEiJk17B89m+W70BJARsQ4eMg9E/c2N9RY6qOX5sjk3WaRgSh2tpjs1ofE7hRhOtDCXlSUoVRRDkGwVB3qS5KiVNZyFz4tO+zNpFvv+6i8IjvuxLm267bmfbCKSF9UDV+JteVjrw+6pvSgqeH7tyzwOike4GpMdvxreO/2Fg6OyryM5OZCZckxOo+sxgBpZdl/i6vZZoAfo+rFZ8QI9Md9m8Z+iior3yX6yfi65lMm3coL/hgDNcYDcYnM38+D9vVnSvDPXqkVkVGBGpwZWGLDq0q7DJrCGrVRNdRG/xmDBQNzkM9NcLFqh0P1A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There is no zsmalloc handle allocation slow path now and writestall is not possible any longer. Remove it from zram_stats. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 3 +-- drivers/block/zram/zram_drv.h | 1 - 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 10e4c6a7735c..07cae8fe9b66 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1437,9 +1437,8 @@ static ssize_t debug_stat_show(struct device *dev, down_read(&zram->init_lock); ret = scnprintf(buf, PAGE_SIZE, - "version: %d\n%8llu %8llu\n", + "version: %d\n0 %8llu\n", version, - (u64)atomic64_read(&zram->stats.writestall), (u64)atomic64_read(&zram->stats.miss_free)); up_read(&zram->init_lock); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 045032acb22f..b7e250d6fa02 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -78,7 +78,6 @@ struct zram_stats { atomic64_t huge_pages_since; /* no. of huge pages since zram set up */ atomic64_t pages_stored; /* no. of pages currently stored */ atomic_long_t max_used_pages; /* no. of maximum pages stored */ - atomic64_t writestall; /* no. of write slow paths */ atomic64_t miss_free; /* no. of missed free */ #ifdef CONFIG_ZRAM_WRITEBACK atomic64_t bd_count; /* no. of pages in backing device */ From patchwork Mon Jan 27 07:29:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13951029 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E745FC0218C for ; Mon, 27 Jan 2025 07:30:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 717352800DA; Mon, 27 Jan 2025 02:30:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6C673280129; Mon, 27 Jan 2025 02:30:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56DC92800DA; Mon, 27 Jan 2025 02:30:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 316212800DA for ; Mon, 27 Jan 2025 02:30:31 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id DC7BF1C99C0 for ; Mon, 27 Jan 2025 07:30:30 +0000 (UTC) X-FDA: 83052409020.19.84974CD Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf30.hostedemail.com (Postfix) with ESMTP id CAC608000F for ; Mon, 27 Jan 2025 07:30:28 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=EKebMEEg; spf=pass (imf30.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737963028; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9PZlb0bpCrEa0EBxHNBCQaxxKLzdyJEYyXpra8I/P5g=; b=J+5un3W0V2ngk8nmD7cUZykX05P6D37aDIyNF9/zRhxJxMjHnm0/NN/OAk7JZvkJXGWCRZ pkhl2TpcrcTL2DfRHkC81FAsdaJb1NKsSZ2rs42G9vRoDBr35IvPcvCJV3HjY/1MoB81BJ MFlkwwyN6oZdRXFjkwiHRykOSplhxSs= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=EKebMEEg; spf=pass (imf30.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.172 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737963028; a=rsa-sha256; cv=none; b=u8TYZ7wMOd2mTPdt6+tWYcypKZJgnWVhDzU/CrqBCLx6kWnbGR5maYiyDvyJTrrUM86s6b MNcZ/d4fU7JJ8tpOHyHaZcZIyNYeBQav99RGcPaRtQPAo04nTscRs9Ehrc5AACOvhSLqGw /4qCciVCmvwbX9HcNEfC9zQB8EKp/7k= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-2167141dfa1so70816315ad.1 for ; Sun, 26 Jan 2025 23:30:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1737963027; x=1738567827; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9PZlb0bpCrEa0EBxHNBCQaxxKLzdyJEYyXpra8I/P5g=; b=EKebMEEgBqMhedceitSCfrDPJTgIke6GGtWEPn/Kmusg75WI3bItLz///Xyo2jl2CR Hm0JcxtXAihsEO8E9DY+jLwh/GHKzfpxk79U3lFlCEiLTmu4FJ2pcCmJ/AAgQB8S0JDU GKXrXLo/e0zkf/2tAjq+w8x052yZdoVsaKY0M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737963027; x=1738567827; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9PZlb0bpCrEa0EBxHNBCQaxxKLzdyJEYyXpra8I/P5g=; b=pAtxWulbEBtp+6jGiG6kIpCNXiKoX5U/IgLeN8RVC2vLciDJiKIzxBxDCKNGnE1EUe 4SSwGrrqqT9fhGzPBv/lrdq4sTi/I9jq6wf3N1PmFHzVrEy3CfDeYSGy4Z5w6Slkn1mU lLtueM6842432sIlzu6G4l+416IrI/FXXcNt4ls0sdhUrUQenT5Y+a3reVzTctwb5F5T Sgjl9vpwqHZuju2qVwHIT/wqfO/fa/8XVzRkcaHkObYVzyRIQx24SuuQQKPQjGVKwB06 EMf1ZWWgIhVcXuNbGcIrwpmbcwIuh7TA0gk2FqxLWZaL7+1d+L2X3SI6Bk28r8a+aGWI lP0g== X-Gm-Message-State: AOJu0Yz7mG8OSY6u1R9ybU3uE1lVzItR1sE5C2QYXMbQCaGXOdLO+j5e bqh1kE3z0mbd9WWJM2PliqjsCECxaj46tUQgqTChdbzv9AD3ewn770UPMoJfgw== X-Gm-Gg: ASbGnctmlFGe+SAeyAZ9LrzGbp25MMx2KtE607AS5LpxG0FIZw8QE3s9K7dzD0a9n5n 7i407shPdfOv/Yv8Tc6wXCVl2qzGQsKvZ5RwdntXeNACSkuRgr5MtGmvs/hjNKMrYqROMMi/Y6T BWBd3SvNOP0+FH6TxokmQwqIEJHexEcBS74Dxw6vR+4ixubFaWb1xkhLcg2Ie8gVkHSclEd3mPL yjDrJqL6/MkYWu3qDnADhIgaDeVolgrmqZh2iHneJ7N0NUYll0hzN2TJuAxx7kfc2NztRKrQ7Ed /u4ulUI= X-Google-Smtp-Source: AGHT+IGG3TeaG4wWi40jK1p6mY3yatjcS1oAjwA+7dpgpn76mSRyGup3rx8p2HazaczZbRSX+r3sbw== X-Received: by 2002:a17:903:40c8:b0:216:6284:8a00 with SMTP id d9443c01a7336-21da4a11a0bmr202401385ad.2.1737963027560; Sun, 26 Jan 2025 23:30:27 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:566d:6152:c049:8d3a]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-21da4141123sm57299285ad.112.2025.01.26.23.30.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 26 Jan 2025 23:30:27 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 9/9] zram: unlock slot during recompression Date: Mon, 27 Jan 2025 16:29:20 +0900 Message-ID: <20250127072932.1289973-10-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250127072932.1289973-1-senozhatsky@chromium.org> References: <20250127072932.1289973-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: CAC608000F X-Stat-Signature: 3yy7jqxdrk7hutiphfjuf7teutzpbnos X-Rspam-User: X-HE-Tag: 1737963028-449195 X-HE-Meta: U2FsdGVkX1/mwpRkNSWuDXFxq0/370jDhCv5Mgm4hFILaXCkphoPVGcHctG7Wt86w8YUD3jFz346Vau0DQ5bL/uHPKKdJkqpNy1P8OXKKrjLh/pc6WqfVq2/9HVb27eWrnenW84EvDY1AUnGMAwmKCqHjCpchJnxDZdUBPXQDd+aLW4PkLylPpYYrGctifuIu1lAGZ4Y21fntmMIvokXsowo7v9MrarnfHquexM+PRE5L7S8c8/4En3tvVTEEIDwzfubc/m9st8JXOMJeIpdcFzoHOJHXhvfcJZV8zbnnEgFAjWVFSVbF/d1nfR4wQLFhEtHXz2nYuDHqAWcpGl05e/sLWXH2UWgaY2KAjpfX7M9A/Ol4cCDK2TuXYuwqHUdTvqishyBPLoPji+kvYsY2roJqRQP3c1nBpnvbkPHatBzSCYdBLt+e1qkXUCEqlA9tpSges+K7xqNynq8N07aI6YGwUZOQ5A3sXcrDDD7cgR/YISTF35W+K7pj8c3vTdnQlmN3Z+zP9kocZeJBkZ0rNn26ERj0O3QwqTTdMigCIkbrdErFgfLGzeHjit+Aa/Dq7ZrwAJZzHJBVMy9xQBqILDH8yl7KQ/aMr8QeGBqeyzmqlYfPAOUUMyJQ3lSih7kLD8TAWWVXOgt+X2zNTmU0Huq7qWdwwO2/F98FhLyep+F5t7HhzuR0dVymT5fqunpLgNkPPwj93VW3G3dj9KUvE2u3StUsZaE6Qwao1jJpOoaaBdau68h3q4Uk02ck9MCZsS32LPzLFHDoB3u4WGwJrh/0TC9n7FImb4LhTTALVB1rQ8iDMSMarf+WkOWuBeIw28eGikFvLPzebMelzzNrrvlP2/UkNrHTYA6SBJ0fC4UXLHZ7NpdROssT9OpkGtUpkpGH3/9hqq1qAFBBAplR+Wydypxupg2v09tG047k1GwBSpOzg1MkjbJkuAQGW9rW21QC0m+Wk+PZlGnbrJ OS0Wddzk iCbpsRQN1hx/6MjWzY16gUsPY42UAdesQlwIqJyaDIEnVCP5Bs/6LdB+b9Tb88SurEj+ADKLn3X8YoXnbh/EugLIeh+HtoHDvptPsk/pTkWDJQv05PcN5RJZGHmUwqu+p408sASXezY8VBjGtrusEctPxvYlNR0zWVNtRumCRsKCpzHEH2C9+Ag1Ut8TpGjB/HDHGwZX8YK8qMe3ns8CJA3AGu2bNgZQBT9B/0Y1l9OL0+iUJSRzqKiagxnf7zZRg1hOr0JECmPl8fX68EA1x/NseGj7gZKtPLlzOrkxsLQ6bA26j3T5X9xCJFWZxNnGjMMcQwv53y9yFgZ/pqe6gbJ6jKnARsR628CnB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Recompression, like writeback, makes a local copy of slot data (we need to decompress it anyway) before post-processing so we can unlock slot-entry once we have that local copy. Unlock the entry write-lock before recompression loop (secondary algorithms can be tried out one by one, in order of priority) and re-acquire it right after the loop. There is one more potentially costly operation recompress_slot() does - new zs_handle allocation, which can schedule(). Release the slot-entry write-lock before zsmalloc allocation and grab it again after the allocation. In both cases, once the slot-lock is re-acquired we examine slot's ZRAM_PP_SLOT flag to make sure that the slot has not been modified by a concurrent operation. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 53 +++++++++++++++++++++++++---------- 1 file changed, 38 insertions(+), 15 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 07cae8fe9b66..ad3e8885b0d2 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1908,6 +1908,14 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, zram_clear_flag(zram, index, ZRAM_IDLE); class_index_old = zs_lookup_class_index(zram->mem_pool, comp_len_old); + + /* + * Set prio to one past current slot's compression prio, so that + * we automatically skip lower priority algorithms. + */ + prio = zram_get_priority(zram, index) + 1; + /* Slot data copied out - unlock its bucket */ + zram_slot_write_unlock(zram, index); /* * Iterate the secondary comp algorithms list (in order of priority) * and try to recompress the page. @@ -1916,13 +1924,6 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (!zram->comps[prio]) continue; - /* - * Skip if the object is already re-compressed with a higher - * priority algorithm (or same algorithm). - */ - if (prio <= zram_get_priority(zram, index)) - continue; - num_recomps++; zstrm = zcomp_stream_get(zram->comps[prio]); src = kmap_local_page(page); @@ -1930,10 +1931,8 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, src, &comp_len_new); kunmap_local(src); - if (ret) { - zcomp_stream_put(zram->comps[prio], zstrm); - return ret; - } + if (ret) + break; class_index_new = zs_lookup_class_index(zram->mem_pool, comp_len_new); @@ -1949,6 +1948,19 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, break; } + zram_slot_write_lock(zram, index); + /* Compression error */ + if (ret) { + zcomp_stream_put(zram->comps[prio], zstrm); + return ret; + } + + /* Slot has been modified concurrently */ + if (!zram_test_flag(zram, index, ZRAM_PP_SLOT)) { + zcomp_stream_put(zram->comps[prio], zstrm); + return 0; + } + /* * We did not try to recompress, e.g. when we have only one * secondary algorithm and the page is already recompressed @@ -1986,17 +1998,28 @@ static int recompress_slot(struct zram *zram, u32 index, struct page *page, if (threshold && comp_len_new >= threshold) return 0; - /* - * If we cannot alloc memory for recompressed object then we bail out - * and simply keep the old (existing) object in zsmalloc. - */ + /* zsmalloc handle allocation can schedule, unlock slot's bucket */ + zram_slot_write_unlock(zram, index); handle_new = zs_malloc(zram->mem_pool, comp_len_new, GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); + zram_slot_write_lock(zram, index); + + /* + * If we couldn't allocate memory for recompressed object then bail + * out and simply keep the old (existing) object in mempool. + */ if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zram->comps[prio], zstrm); return PTR_ERR((void *)handle_new); } + /* Slot has been modified concurrently */ + if (!zram_test_flag(zram, index, ZRAM_PP_SLOT)) { + zcomp_stream_put(zram->comps[prio], zstrm); + zs_free(zram->mem_pool, handle_new); + return 0; + } + dst = zs_map_object(zram->mem_pool, handle_new, ZS_MM_WO); memcpy(dst, zstrm->buffer, comp_len_new); zcomp_stream_put(zram->comps[prio], zstrm);