From patchwork Mon Jan 27 07:29:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13951025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9D76C0218C for ; Mon, 27 Jan 2025 07:30:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3BB4E280125; Mon, 27 Jan 2025 02:30:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 36A672800DA; Mon, 27 Jan 2025 02:30:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20C08280125; Mon, 27 Jan 2025 02:30:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id F3F882800DA for ; Mon, 27 Jan 2025 02:30:08 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 84118C0646 for ; Mon, 27 Jan 2025 07:30:08 +0000 (UTC) X-FDA: 83052408096.25.DC7D96D Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf11.hostedemail.com (Postfix) with ESMTP id B1DA040014 for ; Mon, 27 Jan 2025 07:30:06 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=jK1Yf0Cg; spf=pass (imf11.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.170 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737963006; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QR/p5Tl6lRpVatp+KkNy0La/JT3xx5EKDzU3nlrXbQA=; b=E3mUKwuZtEevtBWELPH6H12Zfl7y/H2DUWvkTxsOU2uG46HCbxitf/nfbLLecrlr07yKCE 2kfi6RuxSyfjk6sBHSgK61/2pH0ph2FMYy5vckBoAjys3N4iB5eFezUn8uCKdm31aqnqbO 3uZDeiR6AIEyW42ZVvWCgqbCKGKcgUA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737963006; a=rsa-sha256; cv=none; b=tViJR0gA3QinUAZ/pjEKcFQsZWbyWTAVtmJ+prCNvn0ptVqlCq1XsLO/tPRtIIzrww1pxt mkQeANzRXyvcz+UutdY7pTuLtEYQ66QMztO9Vn/V0ljN9WoLwX8rdic1bgNbLkegkDs5fU +e6T95nIpWDAIZCaIfVmV0h+KX//xWM= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=jK1Yf0Cg; spf=pass (imf11.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.170 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-21675fd60feso90566205ad.2 for ; Sun, 26 Jan 2025 23:30:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1737963005; x=1738567805; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QR/p5Tl6lRpVatp+KkNy0La/JT3xx5EKDzU3nlrXbQA=; b=jK1Yf0CgpQp4dNQgst6JjqB9bl0i24P7n10IpNzE5OsrRlMNTGgIkhj6Etmenv+g37 //nwkyyApkh5ZuvjWbwc6satfod1i3Ohi+AsMtP2sYfxF2zNnvGOmkYjcWvYiONi6AQk qPLafgAQafE7LWM0p4tMWTP4mWKRcODIisNxs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737963005; x=1738567805; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QR/p5Tl6lRpVatp+KkNy0La/JT3xx5EKDzU3nlrXbQA=; b=YnmsTIxNglcJnRh4XsGw3aWJi2qrfPfixO+JWCIDEERqzHG9BXDzFesLs4qRL65NRy zNJa7Fns6BAlGWlKENYDo2aUNAYmfkOv0BGIAVPaOr0hmOCXL/KuLnrVyrHafG4QWNaO 06iocrR50S3J2EuslGGYVsWIaUImXcB62xhtS0MkBXo9wzpLtJ//TujRPyV5C5jVZmbs XOVygDlvSaAL0M2sFrziRnvoH6TQu9DFegfUG8AfxUj8kQbs4jZfL1Zke7+SajimvKdc f1Y4kZeHxJTcf7lCBZPjOR7Aqp7QwXL/PnNLpfWODocAR1c6Ky56YN67cNdI5EKqCA9L ZWFw== X-Gm-Message-State: AOJu0YyQP2jZ73ONI6CM47IWHTB4xrZr5e0M2QdFHjdKshjRzZuDQTVd u2XO6lwfPmvjWgpeAGPhxuCqHGxLmG44WwWncp+i2KHlv4OGmVSPpivLhuzGrg== X-Gm-Gg: ASbGncvz9oe5YGXdMtD9wo00E4jvs1eREi4PNXVznmffG7lrkRzaAExD7nl3fT0AYBA nzsGuXrFWun2Wt/aQW4ru/pvfhJ3eMJHjqa0xkd+CX8LL2pPyhWoo+Y/+JjBaoAk9E58jmHvdlU Xf77hZT+i1nJCdqSE4gkAR8sIX+Q67hpQd32AsF+Qvo3ASKCfwQGmjP4TBiandigqTbvM2ndzg6 1BP0HB9LLAZO4aHbZHgwGlkrbZYHHRLcgZvLMf+Lkf/feJ0lOP3Ssl5m943A2MnJ/6+3dH39eTk QeRv6AQ= X-Google-Smtp-Source: AGHT+IHV7YMpzbIT6L+DxKKXMn+vhmqmbhrzw8POAb9AfcZ1TJR/LzmJE93baxj7ELaSGxd4dqLq6A== X-Received: by 2002:a05:6a20:8428:b0:1e1:9f57:eaaf with SMTP id adf61e73a8af0-1eb2144d54amr59526793637.6.1737963005514; Sun, 26 Jan 2025 23:30:05 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:566d:6152:c049:8d3a]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72f8a760e1fsm6352930b3a.113.2025.01.26.23.30.03 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 26 Jan 2025 23:30:05 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 5/9] zram: remove two-staged handle allocation Date: Mon, 27 Jan 2025 16:29:16 +0900 Message-ID: <20250127072932.1289973-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250127072932.1289973-1-senozhatsky@chromium.org> References: <20250127072932.1289973-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: B1DA040014 X-Stat-Signature: fogrkqpwypjfh3wpxk7u1z1b4hqxgzkw X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1737963006-364241 X-HE-Meta: U2FsdGVkX1+yENMoH/pZkHz8/T1G6NMRd3M+viDGP8EAw7FE3BjQjyreKpHOsrWv8pkKzOa6arjS5JzbTjiaggp2QpSnInmYa47xzQDhhvav/AxFG5UsT3uk6EMWIVYrB98Unwi/U3fM3o/NUxE2feWU5Af8Axyr3Rd/NVdaEg4/MeiEmBAIUAFzahkRghs6WxSG6htf864OYTF8ookoEmprE9HdR4asHk0IqP1J5KSu7xYG6zEmgE4mAyV4YYCQhfX17y7VSM6+mCWdYJDiNk09efxFdr+YmskX77LTRJTxKpx/VKrya6MTCTmjkwCr+zrolMGoNI2f6VpBLA0m0Jmf1BXRVDTLqmgYHpF3vP0KRrL3kx81/EORMuMSlqpMCjULg2byP8nKN02ZOzHKRiddyQUF9U++tXB9fh/yArKlMnZNo6wG3JAQ2yHlnNn409k6m7tyFcxUle8/QvnCIhjaLVidCDlCfluFJxaYhE0jTldrtwaHWyAJJvX4luGK68NuBiOEmr87ezqpkfIP5qkwiT01abNJ+i2So2IOFfmPiBHV8vxjLy5BHalpgSONm5qjQ52zW3f3YO5r5ShK3DtySif/FGS0RqR1OVPrgiQrTWY/dzyOHqeZw1Of6sbxw5cUhQ+54O+Y4zMqIlciUdJnzTylPqfpUy/bMI+Mai+FR+UXi3mBmTOlbWxVSphNXeZ3/YhpvkOyRuC7E9cI6NggCV3pi9wYncR/PPKsn53Bq7ZGqNdez8q/DWMBkj0Wkx3FLJebXyrcXLPDp70dfBdlbUNsu8cZ1kXFK+hA+ovs8R1Xe0NeUz4u7V0xqbz7f0SfjhO7Opm+pjEQawy27lbzqJAwnED5b6hRndKJ+5y546Ge7AfBgT8ff6dUM4cw0wrKp+LnRyJKWfYjLoDPLpPqnvSjiFMYx3T4goDyFhTMfJ7/+iAPV7I5KaRiLMPaxIIl3jB5DsLzIusNFzz 663qR50H UohUTOvuaaELB2YQ0RlmqjdT9OlD1eaVnDiPapNsZikou96wn3AKub7eDNYO1qYRiyjgdlE3dSHXSDHVqLJfar9aPhZNqJ+kuw0vrp0A8EtCervoyctCOtaRcQRezesQK2+XObQLpRYIAnHHIPOcdaVKygdmaH8SKEszEm6wI2cqRWHgfwDShjopHxJC4MokDUXuX2hsOMoN3BrF2r8MOX8sctQVaxjq22TgFUW7d44JxXSIAy8AGJPQW5WMj4tJELDKeE58REVYQfsG+JhQvRbg3HJs7Jp/2bPJIYn2Ns8oldY3YnIhNWxo/LA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Previously zram write() was atomic which required us to pass __GFP_KSWAPD_RECLAIM to zsmalloc handle allocation on a fast path and attempt a slow path allocation (with recompression) when the fast path failed. Since it's not atomic anymore we can permit direct reclaim during allocation, and remove fast allocation path and, also, drop the recompression path (which should reduce CPU/battery usage). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 41 +++++------------------------------ 1 file changed, 6 insertions(+), 35 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index d867e15d1d2e..046c80aa4310 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1725,11 +1725,11 @@ static int write_incompressible_page(struct zram *zram, struct page *page, static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret = 0; - unsigned long handle = -ENOMEM; - unsigned int comp_len = 0; + unsigned long handle; + unsigned int comp_len; void *dst, *mem; struct zcomp_strm *zstrm; - unsigned long element = 0; + unsigned long element; bool same_filled; /* First, free memory allocated to this slot (if any) */ @@ -1743,7 +1743,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (same_filled) return write_same_filled_page(zram, element, index); -compress_again: zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); mem = kmap_local_page(page); ret = zcomp_compress(zram->comps[ZRAM_PRIMARY_COMP], zstrm, @@ -1752,8 +1751,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (unlikely(ret)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); - pr_err("Compression failed! err=%d\n", ret); - zs_free(zram->mem_pool, handle); return ret; } @@ -1762,36 +1759,10 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return write_incompressible_page(zram, page, index); } - /* - * handle allocation has 2 paths: - * a) fast path is executed with preemption disabled (for - * per-cpu streams) and has __GFP_DIRECT_RECLAIM bit clear, - * since we can't sleep; - * b) slow path enables preemption and attempts to allocate - * the page with __GFP_DIRECT_RECLAIM bit set. we have to - * put per-cpu compression stream and, thus, to re-do - * the compression once handle is allocated. - * - * if we have a 'non-null' handle here then we are coming - * from the slow path and handle has already been allocated. - */ + handle = zs_malloc(zram->mem_pool, comp_len, + GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) - handle = zs_malloc(zram->mem_pool, comp_len, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); - atomic64_inc(&zram->stats.writestall); - handle = zs_malloc(zram->mem_pool, comp_len, - GFP_NOIO | __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) - return PTR_ERR((void *)handle); - - goto compress_again; - } + return PTR_ERR((void *)handle); if (!zram_can_store_page(zram)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm);