From patchwork Fri Jan 31 09:06:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13955132 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F998C0218D for ; Fri, 31 Jan 2025 09:07:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B249D2800F2; Fri, 31 Jan 2025 04:07:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AD3832800EF; Fri, 31 Jan 2025 04:07:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 974FC2800F2; Fri, 31 Jan 2025 04:07:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 79EB22800EF for ; Fri, 31 Jan 2025 04:07:32 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 367361C7B27 for ; Fri, 31 Jan 2025 09:07:32 +0000 (UTC) X-FDA: 83067168744.15.AEDD86E Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf07.hostedemail.com (Postfix) with ESMTP id 4BE904000E for ; Fri, 31 Jan 2025 09:07:30 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=nz3zx3mD; spf=pass (imf07.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738314450; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=loywN5MAU0/S7v3h/0kjXpkRnG0c8JJBzlP3UvQhSOM=; b=2Re9D3Hh4ESCYn9L3zOahfwSMPwoQ9+3uLu47EQ8ykjIWR+6qrmet+jGoOqrx0HsVU4KVj MCrCEhin54Ib5LLOT0lFnetatYh2fWWz+lPCTFaliPpqXCD8U5mSx2f5ho1gLReUdr//xA IfwrePjNWt1JkhvwckD8V2FIv0L9QQc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=nz3zx3mD; spf=pass (imf07.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738314450; a=rsa-sha256; cv=none; b=xYkd3BefBTdM2L1inxRSNxSFzFhPo/5/upXmynnUcG5RxVLhRvfjz/BWSdWLvBRMMMIO+0 q9CNOtYcJSaFzpoEoxjIBiFhW9sEZOfmehvjfPXvlXT5ZC17BSqIipYE1BdtzJ/Tee/qU3 1UMdz3B85gcxQDbAOJ2ou1boZlFvDX0= Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-21670dce0a7so36071665ad.1 for ; Fri, 31 Jan 2025 01:07:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738314449; x=1738919249; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=loywN5MAU0/S7v3h/0kjXpkRnG0c8JJBzlP3UvQhSOM=; b=nz3zx3mDF9XOI1mwl2n1M2PvxeRe695lZamVPuHlno1I0rq2fAidoGbD032NYnM7kA 297yF5q/OgxXjVTnn0d+9+MgR0MCUxC3/ebPnLpW8pYro/FC3q1CcB/1Rzw/hbD1gyLo 2GF3DfI/RfUJVjl+9NWDS2HyE8NV9sIk3dcqY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738314449; x=1738919249; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=loywN5MAU0/S7v3h/0kjXpkRnG0c8JJBzlP3UvQhSOM=; b=OsSEEFDNDCC3NktCld4bVKX/pY4k4mmygNtKzt0O+zMnK9mwv+HZJ9Za5OFQh93XTj zh06KGASfn98uSXaS/NxrmcjdxauVXY6LXBQZg/thzvnUjCPIxDZ3mqOCW7pkVaEDtrs CzexfaDXcvgI/qCLWY+WPFTTPbeLJIrRkdS9q2SXZ9dHabPJyZPLt6cGK+PjDbqCprQF hwqA0GTFxZPEQRUoOApzLRM7WDmAWVTyWRGWCc95x8bPPEOm/AGRvvdCcAUORhGmmxvD 5j2JEse8LiFqnHRWuYPNU19o09c7LoIDFcps3djGHO+cINJWp1pJUCmHmmin2IqQeSQR 2zYg== X-Forwarded-Encrypted: i=1; AJvYcCXYa0oqDnQjZ13N4vFdAAn2ouvqoRXlm0XBEFPETcRBaVKsKlaHTAslBi49YFTW5G0PjPeFByDKbA==@kvack.org X-Gm-Message-State: AOJu0YxEpKQPJqgFii/zOIj10RXgVEbyl0ko2d9ezZkc9sDy/saGN7Xt 7D/JPFbiY80birM3ICq2k0Ua6/QYxkm3YdrSuyuih34OQI81b1DOPWjYw8m4lw== X-Gm-Gg: ASbGnctkRLxKGbYBAn8v47/Kdq2EhKwyRk9YK3zaFfLtRVXNWljjloXLIG2F75rH7Xr jULGoIjZpT+0xkkZ9kA6L8ARlucS1pz5BEZu7msiTqXty3LiktnfrB0f6GdQ3lznWIJUHEZDCMw HB2nKR4fwRpfVQL6bdesXn+0wsIHwmtmEXv2gPYv81L9cHB+q/QIMNFerlgMxmZKJC34hgtyXNN wGSe/F0AJOyrG9jClp/pgSGlxRhVCJtkwY6XMpgH1NdoQG4oQO4B3NhrmJL+K9Tj1ngXHKmqQPo jJJlVio5YvKcCsZ8qg== X-Google-Smtp-Source: AGHT+IF+0FKD5xX6XHXYkP+petrD53hSmc7lcGie6WBqsrwN7A64ICQ6XETnp7Q7BlhjDUtZHY0kTg== X-Received: by 2002:a17:902:cf0a:b0:215:a80b:f6f9 with SMTP id d9443c01a7336-21dd7c449ffmr131180925ad.8.1738314449146; Fri, 31 Jan 2025 01:07:29 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:c752:be9d:3368:16fa]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2f83bcd0d9csm5896824a91.14.2025.01.31.01.07.27 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 31 Jan 2025 01:07:28 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv4 06/17] zram: permit reclaim in zstd custom allocator Date: Fri, 31 Jan 2025 18:06:05 +0900 Message-ID: <20250131090658.3386285-7-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog In-Reply-To: <20250131090658.3386285-1-senozhatsky@chromium.org> References: <20250131090658.3386285-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 4BE904000E X-Stat-Signature: paegjiy11rhisdkxkhc9wxm6741p64ws X-Rspam-User: X-HE-Tag: 1738314450-793991 X-HE-Meta: U2FsdGVkX19/ua9Qeq9GJj9ZqC1L7CEAKCrZyljrAX4Ts83LPZMQCdbwLtNo990rtcu1ktYBYfdxjNmVGZWStwMDKrxMsa/LMBnGBoIC32ruZ23ouEofNZlf3tlvQVIAuSxGxeyOAV6LPg0TMkzkpBAAq83M9lq94RSpzkxE4WhZy7qazBBETy8bsqlzV0aTn6Ei9LVLPlS3gZ5e+UE+IWbcbZnyFU4/QVlAgLyI7BY7Vss8TqXa+hHGtKe6yvwQ6NkNf/sCCzo5KJJcU5q160FpXozbV5v1zsODPDO02StarC3DE9b1kYT7IZVH00NYARJOJ16qYzgDZoBSCWe+9EwA5cXp1b4km+/bVp/cO1bp8AR2pA6a50NjHCxDgK2u08BLKNRbMcpc05L7msdQz0L0rW4sjQR9I95ifqjdQBbhCH4puC08eJ29i/mHnLFz855qMOO7g6R7fjIz3VzuoCcUVY9n67nxNvFuCocQaGrWzP/Z+mhrv8f8MN0IJCWYyveZNBGZW16oQcEaHoQcDUqlzj3XB5bJRK1ve6T2WL9mBPslU2sXw863HLgvsDoOtkBDZkmBYqa1SgsOqvxeBNlKUFDrZGUaoUyDPRtshFFu25mHoY/+rHl/9UH/H+yhu3rtgxO/lWf+MjiKghLYwXxSuoZQqi8h46HBTK6vkd6KOYygLZ+9FjCNbY2tEArKbN7illYse6cv/5ct/YvhDV9nSP7yPzQ+AvSWoxy9B79jMH0AM6kGrNmRPC8mf++LtM5iJhJeGbsK7FrhB79sYVHzA2o20W8gyVYoNjzNlJN45ug2koA+exFy0wU0AxwAAZWlUUjQ4yf6f9zMUQ7n0iUkds51etl/BDiH0KT+K4u7jMWP2hHjFFU/W3bKULKYnNGPscSYoSmWbkxK/UO1/uwF6Cayjap5ARm0wPWkqZu+994i4ra3Q2m11/5bomNrDdWm1oS2rXQ1IqYw5H7 eyrRB+Cc wf6C83ziNdlS7dEoJMlGu+Cl8tWQkKfaVe5GD/jQqnvJDiLD+bvCkSFM5SX7ezbgFQQV7210vP12BdCHawTgvvMg1aiMZwqmi377a9qGT/1lhalD/wHE8Z0wTAQNm067raQUEBTk1mKAVx7sjkXImyuI5oVzmx3KN/Ya4iVLty4+khmCbIfC8/1ZQWrnq+UqtRc1qQaeOElUnlxEoLqocaOlR2jSAaJK9ZKxry8cwPiqqCZzHVo6D0wK6QBcGcAFuaDc+YQCKh6NL9ZAJmbnsA9QBb4bzbz9KO/Hl+HGK3tEhmtXn65lBpp89Hg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When configured with pre-trained compression/decompression dictionary support, zstd requires custom memory allocator, which it calls internally from compression()/decompression() routines. This was a tad problematic, because that would mean allocation from atomic context (either under entry spin-lock, or per-CPU local-lock or both). Now, with non-atomic zram write(), those limitations are relaxed and we can allow direct and indirect reclaim during allocations. The tricky part is zram read() path, which is still atomic in one particular case (read_compressed_page()), due to zsmalloc handling of object mapping. However, in zram in order to read() something one has to write() it first, and write() is when zstd allocates required internal state memory, and write() path is non-atomic. Because of this write() allocation, in theory, zstd should not call its allocator from the atomic read() path. Keep the non-preemptible branch, just in case if zstd allocates memory from read(), but WARN_ON_ONCE() if it happens. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/backend_zstd.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/drivers/block/zram/backend_zstd.c b/drivers/block/zram/backend_zstd.c index 1184c0036f44..53431251ea62 100644 --- a/drivers/block/zram/backend_zstd.c +++ b/drivers/block/zram/backend_zstd.c @@ -24,19 +24,14 @@ struct zstd_params { /* * For C/D dictionaries we need to provide zstd with zstd_custom_mem, * which zstd uses internally to allocate/free memory when needed. - * - * This means that allocator.customAlloc() can be called from zcomp_compress() - * under local-lock (per-CPU compression stream), in which case we must use - * GFP_ATOMIC. - * - * Another complication here is that we can be configured as a swap device. */ static void *zstd_custom_alloc(void *opaque, size_t size) { - if (!preemptible()) + /* Technically this should not happen */ + if (WARN_ON_ONCE(!preemptible())) return kvzalloc(size, GFP_ATOMIC); - return kvzalloc(size, __GFP_KSWAPD_RECLAIM | __GFP_NOWARN); + return kvzalloc(size, GFP_NOIO | __GFP_NOWARN); } static void zstd_custom_free(void *opaque, void *address)