From patchwork Mon Jan 27 07:29:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13951020 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8018EC02192 for ; Mon, 27 Jan 2025 07:29:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EDE322800E7; Mon, 27 Jan 2025 02:29:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E8D182800DA; Mon, 27 Jan 2025 02:29:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D85AA2800E7; Mon, 27 Jan 2025 02:29:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BBD642800DA for ; Mon, 27 Jan 2025 02:29:44 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 70B1B1C8D1C for ; Mon, 27 Jan 2025 07:29:44 +0000 (UTC) X-FDA: 83052407088.20.A21AA1D Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf09.hostedemail.com (Postfix) with ESMTP id 9FB7E14000B for ; Mon, 27 Jan 2025 07:29:42 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="kgk2Un0/"; spf=pass (imf09.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.182 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737962982; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=BsmYbNIEUaThxX01L4uwTC6oHK5caWBItNDD1J/K2nE=; b=c+JfaCPTLuD/pRrcTVq+OvtihqS0C+fEmCLeNPl+TgpQ8l3zn6U7QwXCeYo7Ly077Ab9Yc Dn03NEjB8YvaHMr2TzLmWMZU9325GFaLygfxYG9pNzX+pummBZkCNilwn9H+75C9mMjXlX dxJLWqHUfzemWuOKIJmNfmuqUy8Zk1E= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="kgk2Un0/"; spf=pass (imf09.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.182 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737962982; a=rsa-sha256; cv=none; b=onVB5GwV0648ZJ1BipdDKSuwHCJgqaE4g6SALqNBAiJ8DD6BA3AO90qnHPfGOjotjHCgyM Oe5D6F8FoW7TXk9amwf1hGEhgSqsD0hfM+OgX4xDt+pj/yplLHesYtfKQ+C2sAftUyVr4y 3zRoCbpyx8vqP7m+Ozi5aQIUOW1Cu6Q= Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-21619108a6bso67639695ad.3 for ; Sun, 26 Jan 2025 23:29:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1737962981; x=1738567781; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=BsmYbNIEUaThxX01L4uwTC6oHK5caWBItNDD1J/K2nE=; b=kgk2Un0/TOkKirfoRfB+Def18r6n7UBbjfurZDBCckb48QS2jZ1nfT53r18NQ2bvns lMXb6AFP3LZIGL/D33npZ0m7Z3gKjtcuOcOXuiCeSKEOIPGsp5oJaIFoDsb0cC3odlH9 pp7W9pjiOVzqdf28ZiJtjE7WC+vSbKkErIG30= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737962981; x=1738567781; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=BsmYbNIEUaThxX01L4uwTC6oHK5caWBItNDD1J/K2nE=; b=qU0nXLdi6laZKRWnm/prxNOnqjtTNSJO/cQj7PtvL0sUPVY6mAbEL8DHmDI0CBK8Xp ZTCfpk73VUJJ+sWA0v/k2I7AV67GhBNnLWjE2Uyg8gQ1Tpcltzb3GqbwK0Wqe0pQE0Ch URj+BPR66qN20pUGTS0/BKLbsQwiJiAm5h3oWSWwwpEsL9Ze5xID4pxrarL4qp0PabKm sHnY3HyBdFxqYytMeW32KSqKiocX+2wJK9QXhkHt3LpXwqIs6KW5cTLlBSo6s7STMqM7 H7b4yFMCh9m+kgN04mm2DEJRWDuA9687rhzXEjNhM6hIdKBJl7hcxLUQHuK7iMAebuAf anTw== X-Gm-Message-State: AOJu0YwLabA1U6SaPIj8VX1/yLB7f2xERuNzZQdciM20CzncjZ8prla7 u+FNPKds4g0z+KwHaX0qY/70MEwMjcnhxqbHzdW5/Fucv3YL+Aa2YDTUUU8NKg== X-Gm-Gg: ASbGncuy+beHS7mUi3UsZ0D4ZGFY/b+J0sugQuCtxEX4A1NYySzsECB5BUefZZrd7II 03XEnMEMd0YB/c54k5PvXAH9BE3DTpMdDgJkDr315nCMDWHrWGAeEt+9NcNnVQBxRm3hKUoXPIE MTK3PCPbRlB9iAlWjH+j2XysCe8vQMZevYLeGA0Po8FKUEK13a4rbMT6C5V7+D4OCpmnb6JyWBQ AsSwNRADfL4S1qAOrGUuIBEdiyPxjvYZ9SpQNXLvUc19654vgxmbLs9hNHyt4b1B63Mdf/Bb7Zx Nw1yjvE= X-Google-Smtp-Source: AGHT+IHKje043NX8g76Xo78ig72o7pmbVa/4H2cZkgNkh0Y6172PuHvArbozL9o4cz1qAfpz9BMEAw== X-Received: by 2002:a17:902:f544:b0:216:2a36:5b2e with SMTP id d9443c01a7336-21c355a4cfbmr590906945ad.32.1737962981348; Sun, 26 Jan 2025 23:29:41 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:566d:6152:c049:8d3a]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-21da424ee99sm57004875ad.235.2025.01.26.23.29.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 26 Jan 2025 23:29:40 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 0/9] zram: preemptible writes and occasionally preemptible reads Date: Mon, 27 Jan 2025 16:29:11 +0900 Message-ID: <20250127072932.1289973-1-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog MIME-Version: 1.0 X-Rspamd-Queue-Id: 9FB7E14000B X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: bygaym56bk7c8hapggj1q3mgmrkdi8me X-HE-Tag: 1737962982-514742 X-HE-Meta: U2FsdGVkX1+owWdhNcmUniwE0+kKOjs8ikXGaM7ZU/huWyC6Gxp2c4pBissgKLo26U9/Eg1nh6d3CAg+xQyffJoPP4fskQMF41ZGIA3pCmUnxrJmSqlr6kAD37C6t4/MbNguMJIph9xi/UcWxyzy4jKhCT1KaqUJ0eMgvLGY1QeGb5pjLZkQEkgKPw7ekHDeUW/d3iYpAdxrokMVlnYBrXPV5lpGikHSk8/NtOFYyC2b+GWADPVbkZFHvrjaAGkSSbz9uhPiSnPmDF3W3/0FzsgVPXD9WH9NaiDyZMDyN9NkPonQYGVoGcSNvvBkrxY2wxuhAJe/raCvFQO26pcgGZJrm9t0OBTjJvnXmaTWpaTzo+EWWzPw9TUcvbOxtDO/qIu2W5NK72ZltF6EKX6N+HrnqbA8XUvYPhRD4nJbzjP4iuXrrxJSJQduWzMOiM52kJN4C6FPBzSKrmvnYx8yQNAQj32KGHJka/An47CFV4kdwHnPVRc7DzDkj39pY8dqMgWLEKiFdCGti2eMfzKTo2H8uMw7BbeWJdrRkVKV6fWz+OMBCrgb+hY0bN1QlZ+nGpDece454ngVzLDbozn+UJ/VjBSTv5SoHeBUo28H94TpUoNVti2IDmpAn3g4gOjGJwbSHlXM6cshOEhDW8PyLebvsR+dt4lNkHWLGit8LmUu7A3bg+jtN3EaYtYqmfnEvbjfXzC18M1BR1khzkyWxpSmWdHrWtAKFPjiyD6r97EuzOY/5Kp4mGuU1/BMQ1dlc3INyu2mnZfYht6LFWMJSZHgLBxT76h+LzbgKMHBL/I9I7gxNrC6fS5YW/19zprUwY2LnvgF/txkIUpYpoqAQY3baY5fND8l5a18XwXnmzIsk/O/ZqcZ6FJ9bfWxd/3bcYsXAzjO8cNjlqAOLQiOsVRKWvJ8PTm33Jw8NK9wP/dlPovy/xnTYvr9J6Q1gytNjJ+DvE4c8E9Vktt2z8O k2QUL85K bJzlmyAfnVHx0wtm99xasFLRDIAu+4UYrZ5FZmgn6NDAdSzJkJaAE6pzYpCrI+e98mXfjS9Kwsz5zso9hc7PnUKH2tO8bdmQSWFnUCXEQCTRq95PzbeIjRVFzGjrmAYVDWH07hSMQ3Hv5WBZsN9M406ZaCOqIIq5G5t07KRut89NM2BnahS7P0ANBl2gyi2IptpSb6RLniJNI3Hh4JW90uPQo3UEyZx5K1RzsGeo+zJs1sJQbKY4WFxBsTGhM5DW0MSrzpk5hlW2WIEO3tgEL08fhK8z+IJvgMMGGWdL5KtpIl29BSa2IPZhxHc/P4/NuRfQ8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is Part I (aka Episode IV), which only changes zram and seems like a good start. More work needs to be done, outside of zram, in order to make reads() preemptible in a non-occasional manner. That work will be carried out independently. There are more details in the commit messages, but in short: Currently zram runs compression and decompression in non-preemptible sections, e.g. zcomp_stream_get() // grabs CPU local lock zcomp_compress() or zram_slot_lock() // grabs entry spin-lock zcomp_stream_get() // grabs CPU local lock zs_map_object() // grabs rwlock and CPU local lock zcomp_decompress() Potentially a little troublesome for a number of reasons. For instance, this makes it impossible to use async compression algorithms or/and H/W compression algorithms, which can wait for OP completion or resource availability. This also restricts what compression algorithms can do internally, for example, zstd can allocate internal state memory for C/D dictionaries: do_fsync() do_writepages() zram_bio_write() zram_write_page() // become non-preemptible zcomp_compress() zstd_compress() ZSTD_compress_usingCDict() ZSTD_compressBegin_usingCDict_internal() ZSTD_resetCCtx_usingCDict() ZSTD_resetCCtx_internal() zstd_custom_alloc() // memory allocation Not to mention that the system can be configured to maximize compression ratio at a cost of CPU/HW time (e.g. lz4hc or deflate with very high compression level) so zram can stay in non-preemptible section (even under spin-lock or/and rwlock) for an extended period of time. Aside from compression algorithms, this also restricts what zram can do. One particular example is zram_write_page() zsmalloc handle allocation, which has an optimistic allocation (disallowing direct reclaim) and a pessimistic fallback path, which then forces zram to compress the page one more time. This series changes zram to not directly impose atomicity restrictions on compression algorithms (and on itself), which makes zram write() fully preemptible; zram read(), sadly, is not always preemptible. There are still indirect atomicity restrictions imposed by zsmalloc(). Changing zsmalloc to permit preemption under zs_map_object() is a separate effort (Part II) and will be posted shortly. v1 -> v2: -- reworked slot entry: using atomic_t instead of bucket rwsem locks -- some cleanups (e.g. removal of max_comp_streams which has been deprecated since 2016) Sergey Senozhatsky (9): zram: switch to non-atomic entry locking zram: do not use per-CPU compression streams zram: remove crypto include zram: remove max_comp_streams device attr zram: remove two-staged handle allocation zram: permit reclaim in zstd custom allocator zram: permit reclaim in recompression handle allocation zram: remove writestall zram_stats member zram: unlock slot during recompression Documentation/ABI/testing/sysfs-block-zram | 8 - Documentation/admin-guide/blockdev/zram.rst | 36 +-- drivers/block/zram/backend_zstd.c | 11 +- drivers/block/zram/zcomp.c | 165 ++++++----- drivers/block/zram/zcomp.h | 17 +- drivers/block/zram/zram_drv.c | 310 ++++++++++---------- drivers/block/zram/zram_drv.h | 9 +- include/linux/cpuhotplug.h | 1 - 8 files changed, 266 insertions(+), 291 deletions(-)