From patchwork Thu Jan 30 11:10:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13954486 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5099AC0218D for ; Thu, 30 Jan 2025 11:11:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C9F23280280; Thu, 30 Jan 2025 06:11:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C26D72800D6; Thu, 30 Jan 2025 06:11:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A79FE280280; Thu, 30 Jan 2025 06:11:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 834AC2800D6 for ; Thu, 30 Jan 2025 06:11:56 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 885DE160C55 for ; Thu, 30 Jan 2025 11:11:16 +0000 (UTC) X-FDA: 83063851752.15.95E5136 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf18.hostedemail.com (Postfix) with ESMTP id 9FE191C000D for ; Thu, 30 Jan 2025 11:11:14 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=IGGp3sra; spf=pass (imf18.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.176 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738235474; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=53/T5bAFGFqTW4BKQwy7L5Ysup918Ui5c6VNURPleaw=; b=u2OpnhrYzljyt5X/45izaHCtYn1wPf7gPb+2TTDr7gTgc+AD2NZy7ZnKP+0zk5TywFYkDB wWTCksKDSIll3V9hY8Ua8iZBmnWu8KV49xFqsv9qyz358DFFSeZHLc3uQJFdGuadLiXtHD pq3zbS2YidBXgA5FUMKtXU6YzzLXTX0= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=IGGp3sra; spf=pass (imf18.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.176 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738235474; a=rsa-sha256; cv=none; b=7qrKOlBnhk1KztaiYcP8A5XTO/oVIqKsnze6XsCzfvBopkbcLQwH/rS+Ai26dfU1lsrmWN XwctRBkg9hz+qY0uQbiqH5vwC1zS2oY0c9l01o93eJb5rlvke9wjElcOsZFMtdTxxgmcoC C02Ab4B+OFwtzfnx62ukIPlyFSZctVc= Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2163b0c09afso9874095ad.0 for ; Thu, 30 Jan 2025 03:11:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738235473; x=1738840273; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=53/T5bAFGFqTW4BKQwy7L5Ysup918Ui5c6VNURPleaw=; b=IGGp3sra3igAwRNOgWlpZ8NkloJOHYxk9EwNMqpDjYYGy8Krxd9xol7U361bi6f0ak +xs6HvCepec8TQull+5oLKodAMBPtwvWEwP3Z1MUPWC09D10RWRBczuJ5avH9xmPuOh9 gxpSC+rRWLGiEAM1RG5Iy5uqssBIjwdI4RASM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738235473; x=1738840273; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=53/T5bAFGFqTW4BKQwy7L5Ysup918Ui5c6VNURPleaw=; b=NAvPHIdJpnyE9Y6G/RTj5YryIUo/YgB/1St/ONfE+KfLBbgxSlbkpAEoNGs8i9/wwl 3gkrIo0CjCKBCPoaYSUG/GlQYY9t0pvy3vIETTdmvrIlWIomunb45M8vkii5bao1TA+i Sd4Aj1xxNUNNNb/NnLiFsdyF5AXRYvlWOOgBzIsSMKU/KVOuHc+6LHZPhsds9I1jvflh /YqdHrBIoHXpHufmcNTB9NUUth2r1JB9VBidwvxboBxG5Bgr1oWN29EZ96DImOCjmmyt 1f4t8cGkSm7LOm6r0O/3Xz6QKwSyZ7aWl6zn+qrVoRgQYs80kmH8IEkzSR1Oxe5Iprwd Ho6A== X-Forwarded-Encrypted: i=1; AJvYcCVzOJMtU8yQnV5icYf/5QyIBbMM6JZKgVz1zj0zVyMV46XMTMyJGP2/1AUGWpxFQfXuWh4UMdRCNA==@kvack.org X-Gm-Message-State: AOJu0Yw6TlRVkzVQrIsinLnMC+Vo3ifvoxqAb/DfzLB5JVM/Vvcdk1Ty ozDQMKGPofXtCtfcBjhmvYiw+sXYUv28CcERSLxPQpxslNzOVuhfq/R/cJvEe1XfViaVwDj46HY = X-Gm-Gg: ASbGncvnrf/vkTPhbSQQsKui5sZQb2ZOlXOLzRbyzleecSdD1vlNPhvvDoqrR8DH+Ka C1ChbHItT7FAYjPJ+O/Ljanqdsf+CwSYiGI1cZr0wsGYoN3W1HVTYOuYCCAGnc8gnncQCXDP+xm oMGSdv2X/8E1RR5BR0QLtgD3v+NKOpZupy0F+EWng3CvUMM7R3yVfzYPKfuk5mK0eSrdShnAF1L sTF1BiwD9/NKbjq+JFZyPTxCjx9NpSc3lPywISbL247XmtK8sZuKlefv7XpP8JpFhhZhDghODqc mkYLPyTu/VZi9r8n X-Google-Smtp-Source: AGHT+IHL4Kna9jaxcRu05vdTceS9QAljYz/df3s5STVn/fe4k/r+WgjqGmkxVKfc8QjBK/njaagzow== X-Received: by 2002:a05:6a00:a25:b0:727:99a8:cd31 with SMTP id d2e1a72fcca58-72fd0c148ffmr9136811b3a.14.1738235473252; Thu, 30 Jan 2025 03:11:13 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:d4ce:e744:f46b:4fb]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72fe631bf9asm1165150b3a.27.2025.01.30.03.11.11 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Jan 2025 03:11:12 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv3 00/11] zram: preemptible writes and occasionally preemptible reads Date: Thu, 30 Jan 2025 20:10:45 +0900 Message-ID: <20250130111105.2861324-1-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 9FE191C000D X-Stat-Signature: frn3tp9scbzx6gdco8pswm9fasgbk6dx X-HE-Tag: 1738235474-50880 X-HE-Meta: U2FsdGVkX19bgsf5kYIFWMll0z2bizNSxiW3YUOpAE3ytUgipneBG2guPNz7ixdFXAsof9GpxAsv4TDnRBVvOt4+0VFARjHI60E5WEzW1ESuCuDt35Wg9r3y4T3crqXHA7/0aIa3Y9aS+XeIKav5RRM7FfjEGJtNlJV69OspDWyk+yZ9Z9k6j6Zy4BrW092QLLGRzVf17FYQGb3IkP/MREQ8oWlRnIukXoYY/xYwjIqEYfNdlVkDqV8LY3SsL8hy2I0ETvmfLA9wQRiPn0ycLOad6RnrSNtGXT3N00wu4+ARuX9dZjGRlEwAeBiI5wDYNq/40TezQ01wVg5PJB3gvLBMVz4jJ4+cy4tQ/wMEFWcR1tU7Uh0NOQTySkGxyN7QaSzr8TfpYlOQbkjdX1mXXYUgrYwtcFTrbkjwVcpQR1FkLGGEG3Btm7XSpOOBCYR63lYVEbHGPk4BSfrfhPjPcb3UfP7bgmo1xIk+CnapYzXd24rbeeQMcakEXd/JKlqMZ4jJVVjXbHbc+NfUH2FgLbSCttUdqh/sXWVygwrPxcZlV39u0uFt2cMBeVNe3KEeAL78OOTnWk6kZe4diy4BGgNNpolZ53K5B/HcZmFS3e9/0H3osqxBNTbZ5/keutNVugBss7mcCf7mRS5CnTandAEVlqK0Hd5BnBsmBpH9HtMn5tQuLF5rFbAZqs7KSimXyGEFbW10Rt7JAXM0srrwxOkvgK2Y2SN91nGcFpj+fsUJCDdVC58VxMEn5Oo4/20+Omb6Xhl3e+puDcsB1g9J3D9OjQo6EGrLKGwAXGfg0csKvpAwZzg5lJyhN9S8Dl1b+f/wuDitmW/GclsE+b18Bc4kQk6wc+W0OrBeW7UbMhVFZATD0tzbrioF6qAWPD4uZdPHquF2ddIzB99zCHhEmMrwquuR43vSI0YDpek4Oy/oZZJG9EO1qGECA4rWZFL0x0EZY3QZpsOb53eu2cP K6pP1Ed6 RQ7uAjU7gytTEOnxRRG9qGxyC2grkhBlr5px4NNAZklNlHO3a3VAHrcwzbcagsZwN3Kja36a68/0tEQb+aq9lNr7Nd67eCBzlyIjpdAjH+mtkmJ8Oirm9Huvr76ChGxoLNGrteHc9BmBR1vPFb0doLkFfe+nCahNEktrxrEt6BzvxrNaYk2oiL0FZbXQYY+xR3eUKWm1fsnyVLreNpCgS9iaj0at/Zlr+aSkn15ly6vfRobl89GZGCFf4mtNU5OFbCiulVYsvfe90FIOQf4wTudzp7D84u8HBxPwK5LwU274sjDoqcJ8d4Gs1MXA+GQDBotEYTTsd7MuEiZc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is Part I which only changes zram. In order to make reads() preemptible we also need Part II [1] which changes zsmalloc. There are more details in the commit messages, but in short: Currently zram runs compression and decompression in non-preemptible sections, e.g. zcomp_stream_get() // grabs CPU local lock zcomp_compress() or zram_slot_lock() // grabs entry spin-lock zcomp_stream_get() // grabs CPU local lock zs_map_object() // grabs rwlock and CPU local lock zcomp_decompress() Potentially a little troublesome for a number of reasons. For instance, this makes it impossible to use async compression algorithms or/and H/W compression algorithms, which can wait for OP completion or resource availability. This also restricts what compression algorithms can do internally, for example, zstd can allocate internal state memory for C/D dictionaries: do_fsync() do_writepages() zram_bio_write() zram_write_page() // become non-preemptible zcomp_compress() zstd_compress() ZSTD_compress_usingCDict() ZSTD_compressBegin_usingCDict_internal() ZSTD_resetCCtx_usingCDict() ZSTD_resetCCtx_internal() zstd_custom_alloc() // memory allocation Not to mention that the system can be configured to maximize compression ratio at a cost of CPU/HW time (e.g. lz4hc or deflate with very high compression level) so zram can stay in non-preemptible section (even under spin-lock or/and rwlock) for an extended period of time. Aside from compression algorithms, this also restricts what zram can do. One particular example is zram_write_page() zsmalloc handle allocation, which has an optimistic allocation (disallowing direct reclaim) and a pessimistic fallback path, which then forces zram to compress the page one more time. This series changes zram to not directly impose atomicity restrictions on compression algorithms (and on itself), which makes zram write() fully preemptible; zram read(), sadly, is not always preemptible. There are still indirect atomicity restrictions imposed by zsmalloc(). Changing zsmalloc to permit preemption under zs_map_object() is a separate effort (Part II [1]) and will be posted shortly. [1] https://lore.kernel.org/linux-mm/20250130044455.2642465-1-senozhatsky@chromium.org v2 -> v3: - Fixed a potential double stream-put during recompression (Dan Carpenter) - Added a patch to filter out recompression slots by compression priority - Tweaked cmpxchg loops (based on similar zsmalloc feedback from Uros Bizjak) - Added some more sanity checks to recompression Sergey Senozhatsky (11): zram: switch to non-atomic entry locking zram: do not use per-CPU compression streams zram: remove crypto include zram: remove max_comp_streams device attr zram: remove two-staged handle allocation zram: permit reclaim in zstd custom allocator zram: permit reclaim in recompression handle allocation zram: remove writestall zram_stats member zram: limit max recompress prio to num_active_comps zram: filter out recomp targets based on priority zram: unlock slot during recompression Documentation/ABI/testing/sysfs-block-zram | 8 - Documentation/admin-guide/blockdev/zram.rst | 36 +- drivers/block/zram/backend_zstd.c | 11 +- drivers/block/zram/zcomp.c | 165 +++++---- drivers/block/zram/zcomp.h | 17 +- drivers/block/zram/zram_drv.c | 353 ++++++++++---------- drivers/block/zram/zram_drv.h | 9 +- include/linux/cpuhotplug.h | 1 - 8 files changed, 283 insertions(+), 317 deletions(-)